Method and System for Repairing Reed-Solomon Codes

ABSTRACT

Methods and devices are provided for error correction of distributed data in distributed systems using Reed-Solomon codes. In one embodiment, processes are provided for error correction that include receiving a first correction code for data fragments stored in storage nodes, constructing a second correction code responsive to an unavailable storage node of the storage nodes, performing erasure repair of the unavailable storage node, and outputting a corrected data fragment. The first correction code is a Reed-Solomon code represented as a polynomial and the second correction code is represented as a second polynomial with an increased subpacketization size. Processes are configured to account for repair bandwidth and sub-packetization size. Code constructions and repair schemes accommodate different sizes of evaluation points and provide a flexible tradeoff between the subpacketization size repair bandwidth of codes. In addition, schemes are provided to manage a single node failure and multiple node failures.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Application No.62/855,361 titled METHOD AND SYSTEM FOR REPAIRING REED-SOLOMON CODESfiled on May 31, 2019, the content of which is expressly incorporated byreference in its entirety.

FIELD

The present disclosure generally relates to methods for determiningerror correcting codes for storage systems, distributed storage, andrepairing erasures.

BACKGROUND

Reed-Solomon (RS) codes are very popular in distributed systems becausethey provide efficient implementation and high failure-correctioncapability for a given redundancy level. RS Codes have been used insystems such as Google

s Colossus, Quantcast File System, Facebook

s f4, Yahoo Object Store, Baidu

s Atlas, Backblaze

s Vaults, and Hadoop Distributed File System. However, the transmissiontraffic for repair failures is a huge problem for the application of RScodes.

Existing repair processes for RS codes, such as Shanmugam et al.considered the repair of scalar codes for the first time. Guruswami andWootters (2017) proposed a repair scheme for RS codes. For an RS codewith length n and dimension k over the field F, it achieves the repairbandwidth of n−1 symbols over GF(q). Dau and Milenkovic (2017) improvedthe scheme using full-length codes. Ye and Barg (2016) proposed a schemethat asymptotically approaches the MSR (minimum storage regenerating)bandwidth lower bound of l(n−1)/(n−k) where the sub-packetization sizeis 1=(n−k)n. Tamo et al. (2017) provided an RS code repair schemeachieving the MSR bound and the sub-packetization size is approximatelyn^(n). These two schemes are called MSR schemes.

The repair problem for RS codes can also be generalized to multipleerasures. In this case, the schemes in Dau et al. (2018) and Mardia etal. (2018) work for the full-length code, and Ye et al. (2017) proposeda scheme achieving the multiple-erasure MSR bound. The full-length RScode has high repair bandwidth, and the MSR-achieving RS code has largesub-packetization.

A flexible tradeoff between the sub-packetization size and the repairbandwidth is an open problem of previous schemes. Only the full-lengthRS code with high repair bandwidth and the MSR-achieving RS code withlarge sub-packetization are established. There is a need for providingmore points between the two extremes

the full-length code and the MSR code. One straightforward method is toapply the schemes to the case of l>log_(q) n with fixed (n, k). However,the resulting normalized repair bandwidth grows with l, contradictory tothe idea that a larger l implies smaller normalized bandwidth.

There is a desire for improvements in repair bandwidths and processesand configurations for RS code repair.

BRIEF SUMMARY OF THE EMBODIMENTS

Disclosed and claimed herein are systems and methods for efficientrepair of Reed-Solomon codes. In one embodiment, a method for errorcorrection of distributed data includes receiving, by a device, a firstcorrection code for a plurality of data fragments. These fragments arestored in a plurality of storage nodes. The first correction code is aReed-Solomon code having a data symbol for each the plurality of storagenodes. Moreover, this correction code is represented as a firstpolynomial over a first finite field and has a first subpacketizationsize. The method also includes constructing, by the device, a secondcorrection code in response to at least one unavailable storage node ofthe plurality of storage nodes. The second correction code isrepresented as a second polynomial over a second finite field and has anincreased subpacketization size relative to the first polynomial. Themethod includes performing, by the device, erasure repair for the atleast one unavailable storage node using the second correction code. Thesecond correction code is applied to available data fragments of theplurality of data fragments for erasure repair. In one embodiment, theerasure repair utilizes at least one coset of a multiplicative group ofthe second finite field. The method also includes outputting, by thedevice, a corrected data fragment based on the erasure repair.

In one embodiment, the second correction code is constructed is forsingle erasure repair. The erasure repair utilizes at least one of onecoset and two cosets. In this embodiment, the code length (n) anddimension (k) of the first correction code are maintained.

In one embodiment, the second correction code is constructed for singleerasure repair. The erasure repair utilizes multiple cosets. In thisembodiment, the code length (n) of the second correction code isincreased and redundancy (r) is fixed.

In one embodiment, the second correction code is constructed for singleerasure repair. In this embodiment, the second correction code providesa scalar code with evaluation points selected from one coset.

In one embodiment, the second correction code is constructed for singleerasure repair with evaluation points in selected two cosets. In thisembodiment, the erasure repair includes selection of correction codepolynomials that have a full rank condition in a coset of an unavailablenode and rank 1 when evaluated at another coset.

In one embodiment, the second correction code is constructed for singleerasure repair. In this embodiment, the evaluation points for erasurerepair are chosen from multiple cosets to increase code length.

In one embodiment, the second correction code is constructed forrepairing multiple erasures in the plurality of data fragments. In thisembodiment, the erasure repair includes use of a helper node toreconstruct at least one symbol for a first data fragment. Moreover, inthis embodiment, the reconstructed symbol is used for erasure error of asecond data fragment.

In one embodiment, the erasure repair includes using a first dualcodeword and a second dual codeword as repair polynomials. Additionally,this embodiment includes combining a trace function with dual codewordsin order to generate fragments.

In one embodiment, the method for error correction of distributed dataas described above further includes determining an erasure repair schemefor the plurality of data fragments.

Another embodiment is directed to a device configured for errorcorrection of distributed data that includes modifying a Reed-Solomoncorrection code. The device includes an interface configured to receivea first correction code for a plurality of data fragments stored in aplurality of storage nodes. In this embodiment, the correction code is aReed-Solomon code having a data symbol for each the plurality of storagenodes. Moreover, the first correction code is represented as a firstpolynomial over a first finite field and has a first subpacketizationsize. The device further includes a repair module, coupled to theinterface. The repair module is configured to construct a secondcorrection code in response to at least one unavailable storage node ofthe plurality of storage nodes. In this embodiment, the secondcorrection code is a second polynomial representation over a secondfinite field. The second polynomial representation has an increasedsubpacketization size relative to the first polynomial. The repairmodule is further configured to perform erasure repair for the at leastone unavailable storage node using the second correction code. Thesecond correction code is applied to available data fragments of theplurality of data fragments for erasure repair. The erasure repairutilizes at least one coset of a multiplicative group of the secondfinite field. In this embodiment, the repair module is furtherconfigured to output a corrected data fragment based on the erasurerepair.

BRIEF DESCRIPTION OF THE DRAWINGS

The features, objects, and advantages of the present disclosure willbecome more apparent from the detailed description set forth below whentaken in conjunction with the drawings in which like referencecharacters identify correspondingly throughout and wherein:

FIG. 1 illustrates an exemplary network architecture for performingerror correction of distributed data according to one or moreembodiments;

FIG. 2 depicts a process for error correction of distributed dataaccording to one or more embodiments;

FIG. 3 depicts a diagram of a computing device that may be configuredfor error correction of distributed data according to one or moreembodiments;

FIG. 4 illustrates a graphical representation showing normalized repairbandwidth values that correspond to various subpacketization sizes as aresult of implementing different repair schemes according to one or moreembodiments;

FIG. 5 illustrates a graph representation showing normalized repairbandwidth values that correspond to various subpacketization sizes as aresult of implementing different repair schemes according to one or moreembodiments;

FIG. 6 shows erasure locations in various data packets according to oneor more embodiments;

FIG. 7 illustrates a graph representation showing normalized repairbandwidth values that correspond to various subpacketization sizes as aresult of implementing different repair schemes in accordance with oneor more embodiments; and

FIG. 8 illustrates a process for error correction of distributed dataaccording to one or more embodiments that includes modifying aReed-Solomon correction code.

DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENTS Overview andTerminology

One aspect of the disclosure is directed to determining error correctingcodes for storage systems, distributed storage, and repairing erasure.

In distributed storage, storage nodes may be unavailable for data accessdue to node failure or temporary high workload. To alleviate this issue,erasure correcting codes are applied to a plurality of storage nodes fora plurality of data fragments. Each storage node may correspond to onesymbol of the erasure correcting code (e.g., codeword). Reed-Solomon(RS) codes are one of the most widely used codes. In particular, for $k$information nodes, $n−k$ redundant parities nodes are added and RS codescan tolerate $n−k$ unavailable nodes. However, when node unavailabilityoccurs, a lot of communication traffic is incurred in order to repairthe information. Schemes that require the traffic equal to the contentof $k$ storage nodes are prohibitive in large-scale storage systems.Embodiments herein provide methods/algorithms to implement the repair ofthe RS codes that require less transmission (i.e., less repairbandwidth) compared to existing repair codes. Moreover, methodsdescribed here achieve low complexity for a given bandwidth sincesub-packetization size is small, and account for bandwidthconsiderations.

In distributed storage, every code word symbol corresponds to a storagenode, and communication costs between storage nodes need to beconsidered when node failures are repaired. The repair bandwidth isdefined as the amount of transmission required to repair a single nodeerasure, or failure, from the remaining nodes (called helper nodes). Foran RS code over the finite field F=GF(q^(l)), the sub-packetization sizeof the code is 1. Here q is a prime number, and the finite field mayrelate to a Galois field. As used herein erasure may include one or moreof lost, corrupt and unavailable code fragments. Small repair bandwidthis desirable to reduce the network traffic in distributed storage. Smallsub-packetization is attractive due to the complexity in fieldarithmetic operations. Embodiments discussed herein provideconstructions and repair algorithms of RS codes to provide a flexibletradeoff between the repair bandwidth and the sub-packetization size.

Erasure codes are ubiquitous in distributed storage systems because theycan efficiently store data while protecting against failures.Reed-Solomon (RS) code is one of the most commonly used codes because itachieves the Singleton bound and has efficient encoding and decodingmethods. Codes matching the Singleton bound are called maximum distanceseparable (MDS) codes, and they have the highest possiblefailure-correction capability for a given redundancy level. Indistributed storage, every code word symbol corresponds to a storagenode, and communication costs between storage nodes need to beconsidered when node failures are repaired. Repair bandwidth of RScodes, may be defined as the amount of transmission required to repair asingle node erasure, or failure, from all the remaining nodes (calledhelper nodes). For a given erasure code, when each node corresponds to asingle finite field symbol over F=GF(q^(l)), the code is said to bescalar; when each node is a vector of finite field symbols in B=GF(q) oflength l, it is called a vector code or an array code. In both cases, wesay the sub-packetization size of the code is l. Here q is a power of aprime number. Compared to considering the repair of scalar codes for thefirst time and a recent repair scheme for RS codes where the key idea isthat: rather than directly using the helper nodes as symbols over F torepair the failed node, one treats them as vectors over the subfield B.Thus, a helper may transmit less than l symbols over B, resulting in areduced bandwidth. For an RS code with length n and dimension k over thefield F, denoted by RS(n, k), a prior approach achieves a repairbandwidth of n

1 symbols over B. Moreover, when n=q^(l) (called the full-length RScode) and n

k=q^(l−1) the scheme provides the optimal repair bandwidth. An improvedscheme such that the repair bandwidth is optimal for the full-length RScode and any n

k=q^(s), 1≤s≤log(n

k). In one embodiment, the help nodes are treated as vectors over asubfileld B, in that the helper stores an element x∈F. Any element x∈Fcan be presented as x=x₀β₀+x₁β₁+ . . . +x_(l−1)β_(l−1) with x₀, x₁, . .. , x_(l−1) ∈B and a set of basis {β₀,β₁, . . . , β_(l−1)} for F over B.For example, x∈F=GF(2²)={0,1, α,α²} can be presented as x=x₀β₀+x₁β₁ withx₀, x₁ ∈B=GF(2)={0,1} and the basis {β₀=1, β₁=α}. Here F is extendedfrom B by a that is a root of the polynomial x²=1+x. Elements includedin the subfield B may be the elements in the Galois field B=GF(q). Hereare some examples. If q=2, the subfield B=GF(2)={0,1}. If q is a primenumber, the subfield is B=GF(q)={0,1, . . . , q−1}. If q=4, the subfieldis B=GF(2²) {0,1, α,α²}. The relationship between the subfield B and thefinite field F may be that F is the Galois field extended from B by amonic irreducible polynomial of degree 1. All elements of B arecontained in F. See the previous example of F=GF(2²)={0,1, α,α²} andB=GF(2)={0,1}.

For the full-length RS code, some schemes are optimal for singleerasure. However, the repair bandwidth of these schemes still has a biggap from the minimum storage regenerating (MSR) bound. In particular,for an arbitrary MDS code, the repair bandwidth b, measured in thenumber of symbols over GF(q), is lower bounded by

$\begin{matrix}{b \geq {\frac{\left( {n - 1} \right)}{n - k}.}} & (1)\end{matrix}$

An MDS code satisfying the above bound is called an MSR code. In fact,most known MSR codes are vector codes. For the repair of RS codes,proposed schemes that asymptotically approach the MSR bound as n growswhen the sub-packetization size is

=(n

k)_(n). A technique provided an RS code repair scheme achieving the MSRbound when the sub-packetization size is l=n^(n).

The repair problem for RS codes can also be generalized to multipleerasures. In this case, the schemes work for the full-length code andfor centralized repair. According to one embodiment, a scheme isprovided for achieving the multiple-erasure MSR bound.

The need for small repair bandwidth is motivated by reducing the networktraffic in distributed storage, and the need for the smallsub-packetization is due to the complexity in field arithmeticoperations. It is demonstrated that the time complexity ofmultiplications in larger fields are much higher than that of smallerfields. Moreover, multiplication in Galois fields are usually done bypre-computed look-up tables and the growing field size has a significantimpact on the space complexity of multiplication operations. Largerfields require huge memories for the look-up table. For example, in GF(2¹⁶), 8 GB are required for the complete table, which is impractical inmost current systems. Some logarithm tables and sub-tables are used toalleviate the memory problems for large fields, while increasing thetime complexity at the same time. For example, in the Intel SIMDmethods, multiplications over GF (2¹⁶) need twice the amount ofoperations as over GF (2⁸), and multiplications over GF (2³²) need 4times the amount of operations compared to GF (2⁸), which causes themultiplication speed to drop significantly when the field size grows.

To illustrate the impact of the sub-packetization size on thecomplexity, an encoding example is provided. To encode a single paritycheck node, k multiplications and k additions need to be performed overGF(q^(l)). For a given systematic RS (n, k) code over GF(q^(l)), we canencode kl log₂ q bits of information by multiplications of (n

k)kl log₂ q bits and additions of (n

k)kl log₂ q bits. When M bits are encoded into RS (n k) codes, we needM/(kl log₂ q) copies of the code and we need multiplications of M(n

k) bits and additions of M(n

k) bits in GF(q^(l)) in total. Although the total amount of bits we needto multiply is independent of l, the complexity over a larger field ishigher in both time and space. For a simulation of the RS code speedusing different field sizes on different platforms, RS codes may havefaster implementation in both encoding and decoding for smaller fields.

Besides complexity, the small sub-packetization level also has manyadvantages such as easy system implementation, great flexibility andbandwidth-efficient access to missing small files, which makes itimportant in distributed storage applications.

As can be seen from the two extremes, a small sub-packetization levelalso means higher costs in repair bandwidth, and not many other codesare known besides the extremes. For vector codes, existing systems mayprovide small sub-packetization codes with small repair bandwidth, butonly for single erasure or also present a tradeoff between thesub-packetization level and the repair bandwidth for the proposedHashTag codes implemented in Hadoop. For scalar codes, an MSR scheme hasbeen extended to a smaller sub-packetization size, but it only works forcertain redundancy r and single erasure.

Embodiments herein are directed to a design for three single-erasure RSrepair schemes, using the cosets of the multiplicative group of thefinite field F. Note that the RS code can be viewed as n evaluations ofa polynomial over F. The evaluation points of the three schemes are partof one coset, of two cosets, and of multiple cosets, respectively, sothat the evaluation point size can vary from a very small number to thewhole field size. In the schemes designed in this paper, parameter isprovided that can be tuned, and provides a tradeoff between thesub-packetization size and the repair bandwidth.

According to one embodiment, a first scheme is provided for an RS (n, k)code that achieves the repair bandwidth

$\frac{}{a}\left( {n - 1} \right)\left( {a - s} \right)$

for some a, s such that n>q^(a), r

n−k>q* and a divides l. Specifically, for the RS (14, 10) code, weachieve repair bandwidth of 52 bits with l=8, which is 35% better thanthe naive repair scheme.

According to another embodiment, a second scheme reaches the repairbandwidth of

$\left( {n - 1} \right)\frac{ + a}{2}$

for some such that n≥2(q^(a)−1), a divides l and

$\frac{}{r} < {a.}$

According to another embodiment, a third scheme attains the repairbandwidth of

${\frac{}{r}\left( {n + 1 + {\left( {r - 1} \right)\left( {q^{a} - 2} \right)}} \right)\mspace{14mu} {when}\mspace{14mu} n} \leq {\left( {q^{a} - 1} \right)\log_{r}{\frac{}{a}.}}$

Another realization of the third scheme attains the repair bandwidth of

$\frac{}{r}\left( {n - 1 + {\left( {r - 1} \right)\left( {q^{a} - 2} \right)}} \right)$

where

$ \approx {{a\left( \frac{n}{q^{a} - 1} \right)}^{(\frac{n}{q^{a} - 1})}.}$

The second realization can also be generalized to any d helpers, fork≥d≥n−1.

Embodiments provide characterizations of linear multiple-erasure re-pairschemes, and propose two schemes for multiple erasures, where theevaluation points are in one coset and in multiple cosets, respectively.Again, the parameter a is tunable. According to one embodiment, a is aparameter that may be selected to help reduce the repair bandwidth foran RS(n, k) over F. As long as the parameter a satisfies the conditionin paragraph [0041], a repair scheme can be provided to reach the repairbandwidth in paragprah [0041]. For example, we can choose a=4 and getrepair bandwidth of 52 bits for RS(14,10) over GF(2⁸).

According to one aspect, proof is provided that any linear repair schemefor multiple erasures in a scalar MDS code is equivalent to finding aset of dual codewords satisfying certain rank constraints.

For an RS (n, k) code with

$e < {\frac{1}{a - s}\sqrt{{\log \;}_{q}n}}$

erasures, our first scheme achieves the repair bandwidth

$\frac{e\; }{a}\left( {n - e} \right)\left( {a - s} \right)$

for some a, s such that n<q^(a), r=n−k>q^(s) and a divides l.

For an RS (n, k) code, our second scheme works for e≤n−k erasures and ne helpers. The repair bandwidth depends on the location of the erasuresand in most cases, we achieve

${\frac{e\; }{d - k + e}\left( {n - e + {\left( {n - k + e} \right)\left( {q^{a} - 2} \right)}} \right)\mspace{14mu} {where}\mspace{14mu} } \approx {a\left( \frac{n}{q^{a} - 1} \right)}^{(\frac{n}{q^{a} - 1})}$

and a divides l. We demonstrate that repairing multiple erasuressimultaneously is advantageous compared to repairing single erasuresseparately.

According to one aspect of the disclosure, processes and configurationsare provided for less repair bandwidth, and less complexity for a givenbandwidth. These processes may account for the tradeoff betweencomplexity and repair bandwidth. The processes apply to any RS codeparameters.

As used herein, the terms

a

or

an

shall mean one or more than one. The term

plurality

shall mean two or more than two. The term

another

is defined as a second or more. The terms

including

and/or

having

are open ended (e.g., comprising). The term

or

as used herein is to be interpreted as inclusive or meaning any one orany combination. Therefore,

A, B or C

means

any of the following: A; B; C; A and B; A and C; B and C; A, B and C

. An exception to this definition will occur only when a combination ofelements, functions, steps or acts are in some way inherently mutuallyexclusive.

Reference throughout this document to

one embodiment,

certain embodiments,

an embodiment,

or similar term means that a particular feature, structure, orcharacteristic described in connection with the embodiment is includedin at least one embodiment. Thus, the appearances of such phrases invarious places throughout this specification are not necessarily allreferring to the same embodiment. Furthermore, the particular features,structures, or characteristics may be combined in any suitable manner onone or more embodiments without limitation.

Exemplary Embodiments

FIG. 1 illustrates an exemplary network architecture according to oneembodiment for error correction of distributed data. This exemplarynetwork architecture may include efficient repair of Reed-Solomon codes.Storage in a distributed storage system (DSS) often requires erasureencoding. System 100 relates an exemplary distributed storage system. Asdiscussed herein, systems and processes are provided for determiningrepair codes for one or more network applications. As shown in FIG. 1,system 100 includes a storage controller 105, plurality of storagearrays 110 _(1-n), client devices 115 _(1-n), and repair device 120.System 100 can includes a plurality of components and/or devicesrelative to a communication network 125. System 100 may be configured toprovide one or more applications for more or more network services andnetwork enterprises. Applications and enterprises may be executed byclient devices 115 _(1-n) and storage controller 105 may be configuredto store data for the enterprises and applications to one or morestorage arrays 110 _(1-n). According to one embodiment, repair device120 maybe configured to determine repair codes for data stored by system100. According to another embodiment, one or elements of system 100 mayinclude functional modules and/or components for determining erasurecodes.

Elements of FIG. 1 are exemplary and system 100 may include one or moreother components of elements including one or more servers, firewalls,routers, switches, security appliances, antivirus servers, or otheruseful network devices, along with appropriate software.

In distributed storage, storage nodes, such as storage arrays 110 _(1-n)may be unavailable for data access due to node failure or temporary highworkload. To alleviate this issue, erasure correcting codes are appliedand each storage node corresponds to one symbol of the codeword.

Embodiments described herein are directed to processes usingReed-Solomon (RS) codes for storage. Erasure encoding may be used fordata structures mathematically transformed into n different fragments.One or more original data structures may also be referred to as a fileor data. Operations described herein may be directed to one or more datastructures and file types. Coded fragments or fragments can include anysuitable piece of a file from which the full file may be reconstructed(alone or in conjunction with other fragments), including the formalpieces of a file yielded by the erasure coding technique. Fragments maybe referred to as (n,k) Coding. For example in the case where n is 5 andk is 4, the original data structure might be stored with one fragment oneach of 5 storage nodes, and if any one of the storage nodes fail, it ispossible to reconstruct the original file from any remaining fragments.When a node failure occurs, it may be desirable to reconstruct theerasure encoded file, so that once again the full n fragments areavailable for redundancy.

A plurality of repair schemes are provided. The comparison of schemesdescribed herein, as well as the comparison to previous works, are shownin Tables I and II, and are discussed in more details below.

FIG. 2 depicts a process 200 for error correction of distributed databased on modifying a Reed-Solomon correction code according to one ormore embodiments. Process 200 may be employed by a device, such asrepair device (e.g., repair device 120) of a system (e.g., system 100)and one or more other components. According to one embodiment, process200 may be initiated upon a device (e.g., repair device 120) receiving afirst correction code at block 205. The device may receive the firstcorrection code from one or more client devices (e.g., one or more ofclient devices 115 _(1-n)) via the communication network 125. In oneembodiment, the first correction code is received for a plurality ofdata fragments that are stored in a plurality of storage nodes.Additionally, in one embodiment, the first correction code is aReed-Solomon code having a data symbol for each of the plurality ofstorage nodes. The first correction code is represented as a polynomialover a first finite field and has a first subpacketization size. In oneembodiment, a finite field has a property such that calculationsperformed using one or more elements of the field always results in anelement with a value that is within the field.

At block 210, process 200 includes constructing a second correction codein response to at least one unavailable storage node of the plurality ofstorage nodes and/or unavailability one or more data fragments. Thesecond correction code may be represented as a second polynomial over asecond finite field and has an increased subpacketization size relativeto the first polynomial. In one embodiment, the second correction codeis a new Reed-Solomon code constructed using the first correction code.In one embodiment, the unavailability of the storage node may be due toan erasure error in one of the plurality of data fragments stored in theplurality of storage nodes. An erasure error may occur if a storagelocation of a particular symbol included in one of the data fragmentsstored in the storage nodes is unknown. In another embodiment, thesecond correction code is constructed for single erasure repair. In oneexample of the construction of the second correction code for singleerasure repair, the erasure repair utilizes at least one of one cosetand two cosets. In this example, the code length (n) and dimension (k)of the first correction code are maintained. In another example of theconstruction of the second correction code for single erasure repair,the second correction utilizes multiple cosets. In this example, thecode length (n) of the second correction code is increased andredundancy (r) is fixed. In another example of the construction of thesecond correction code for single erasure repair, the second correctioncode provides a scalar code with evaluation points selected from onecoset.

According to another example of the construction of the secondcorrection code for single erasure repair, the second correction code isconstructed with evaluation points that are selected in two cosets. Inthis example, the erasure repair includes selection of correction codepolynomials that have a full rank condition in a coset of an unavailablenode and a rank 1 when evaluated at another coset. In another example ofthe construction of the second correction code for single erasurerepair, evaluation points for erasure repair are chosen from multiplecosets to increase code length.

In another embodiment, the second correction code is constructed forrepairing multiple erasures in the plurality of fragments. In oneexample of the construction of the second correction code for multipleerasure repair, the erasure repair includes use of a helper node toreconstruct at least one symbol for a first data fragment. In thisexample, the reconstructed symbol is used for erasure repair of a secondfragment. The second correction code (e.g., a new Reed-Solomon code) canbe viewed as n evaluations of a polynomial over F. These evaluationpoints are part of one coset, of two cosets, and of multiple cosets,respectively. As such, the evaluation point size can vary from a verysmall number to the whole field size.

At block 214, process 200 may optionally determined an erasure repairscheme for the plurality of data fragments. In one embodiment, thedetermined erasure scheme may be used to repair a single erasure ormultiple erasures in using the constructed second correction code.

At block 215, erasure error of the unavailable storage node is performedusing the second correction code. In one embodiment, the secondcorrection code is applied to available data fragments of the pluralityof data fragments for erasure repair. The erasure error utilizes atleast one coset of a multiplicative group of the second finite field. Inone embodiment, repairing the erasure includes using a first dualcodeword and a second dual codeword as repair polynomials. In thisembodiment, a trace function is combined with the dual code words togenerate fragments. In one embodiment, the trace function is used toobtain subfield symbols, as described in the

Preliminaries

section below. The use of the trace function in combination with dualcodewords is also discussed in the

Reed-Solomon Repair Schemes for Multiple Erasures

and

REPAIR ALGORITHM FOR RS (14, 10) CODE

sections.

According to one embodiment, erasure repair at block 215 includes afirst scheme is provided for an RS (n, k) code that achieves the repairbandwidth

$\frac{}{a}\left( {n - 1} \right)\left( {a - s} \right)$

for some a, s such that n<q^(a)

n−k>q^(s) and a divides l. Specifically, for the RS (14, 10) code, weachieve repair bandwidth of 52 bits with l=8, which is 35% better thanthe naive repair scheme.

According to another embodiment, erasure repair at block 215 includes asecond scheme that reaches the repair bandwidth of

$\left( {n - 1} \right)\frac{ + a}{2}$

for some a such that n≤2(q^(a)−1), a divides l and

$\frac{}{r} < {a.}$

According to another embodiment, erasure repair at block 215 includes athird scheme that attains the repair bandwidth of

$\frac{}{r}\left( {n + 1 + {\left( {r - 1} \right)\left( {q^{a} - 2} \right)}} \right)$

when

$n \leq {\left( {q^{a} - 1} \right)\log_{r}{\frac{}{a}.}}$

Another realization of the third scheme attains the repair bandwidth of

$\frac{}{r}\left( {n - 1 - {\left( {r - 1} \right)\left( {q^{a} - 2} \right)}} \right)$

where

$ \approx {{a\left( \frac{n}{q^{a} - 1} \right)}^{(\frac{n}{q^{a} - 1})}.}$

The second realization can also be generalized to any d helpers, fork≤d≤n−1.

Embodiments provide characterizations of linear multiple-erasure re-pairschemes, and propose two schemes for multiple erasures, where theevaluation points are in one coset and in multiple cosets, respectively.Again, the parameter a is tunable.

Process 200 may provide linear repair of RS codes. As used herein, forpositive integer i, we use [i] to denote the set {1, 2, . . . , i}. Forintegers a, b, we use a I b to denote that a divides b. For real numbersan, bn, which are functions of n, we use a≈b to denote

${\lim_{n->\infty}\frac{a_{n}}{b_{n}}} = 1.$

For sets A⊆B, we use B/A to denote the difference of A from B. For afinite field F we denote by

=

{0} the corresponding multiplicative group. We write

≤

for

being a subfield of

. For element β∈F and E as a subset of F, we denote βE={βs, ∀s∈E}. A^(T)denotes the transpose of the matrix A.

At block 216, process 200 includes outputting a corrected data based onthe erasure repair that is performed in block 215. In one embodiment,errors in one or more of the plurality of data fragments are correctedand the data fragments are restored in their entirety such thatinformation associated with all data fragments stored in the storagenodes are accessible and known. Output of data can include output of acorrected fragment.

According to one embodiment, erasure correcting codes in erasure schemesdetermined by process 200 are applied with respect to each storage nodecorresponding to one symbol of the codeword. Similarly, erasurecorrecting codes determined by process 200 may be for storage systems,distributed storage, and repairing erasures. In one embodiment, process200 also includes a parameter that can be tuned, and provides a tradeoffbetween the sub-packetization size and the repair bandwidth.

FIG. 3 depicts a diagram of a computing device 300 that may beconfigured for error correction of distributed data based on modifying aReed-Solomon correction code according to one or more embodiments. Unit300 includes processor 305, memory 310, and input/output interface 315.In some embodiments, unit 300 may also include a repair module 320. Incertain embodiments, repair module may relate to functional element ofprocessor 305. Unit 300 may be configured to receive data from one ormore devices.

Processor 305 may be configured to provide one or more repair functions,including determine one or more of a single erasure, and multipleerasures. According to one embodiment, processor 305 is configured toperform one or more operations. Memory 310 may include ROM and RAMmemory for operation of unit 300 and processor 305. Input/outputinterface 315 may include one or more inputs or controls for receivingand providing data.

In one embodiment, input/output interface 315 may be configured toreceive a first correction code for a plurality of data fragments thatare stored in a plurality of storage nodes. In one embodiment, the firstcorrection code is a Reed-Solomon code that has a data symbol for eachof the plurality of storage nodes and is represented as a polynomialover a first finite field with a first subpacketization size. As statedabove, in one embodiment, a finite field has a property such thatcalculations performed using one or more elements of the field alwaysresult in an element having a value that is within the field.

According to one embodiment, the repair module 320 may be coupled to theinput/output interface 315. In this embodiment, the repair module 320may be configured to construct a second correction code in response toat least one unavailable storage node of the plurality of storage nodes.The second correction code is represented as a second polynomial over asecond finite field and has an increased subpacketization size relativeto the first polynomial. In one embodiment, the second correction codeis a new Reed-Solomon code. As stated above, the unavailability may bedue to an erasure error in one of the plurality of data fragments. Anerasure error may occur if a storage location of a particular symbolincluded in one of the data fragments is unknown. The repair module 320may construct a second correction code using one or more operationsdiscussed relative to process 200 of FIG. 2.

In one embodiment, the repair module 320 may be configured to performerasure repair in the unavailable storage node using the secondcorrection code. In one embodiment, the second correction code isapplied to available data fragments of the plurality of data fragmentsfor erasure repair. The erasure error utilizes at least one coset of amultiplicative group of the second finite field. The repair module 320may perform erasure repair using one or more operations discussedrelative to process 200 of FIG. 2. In yet another embodiment, repairmodule 320 may be configured to output a corrected data fragment basedon the erasure repair. In one embodiment, as stated above, errors in oneor more of the plurality of data fragments are corrected and the datafragments are restored in their entirety such that all data fragmentsstored in the storage nodes are accessible and known.

Preliminaries

In this section, a review of linear repair scheme of RS code and a basiclemma used in our proposed schemes is provided.

The Reed-Solomon code RS (A, k) over

=GF(

) of dimension k with n evaluation points A={α1, α2, . . . , αn}⊆F isdefined as

RS(A,k)={(f(α₁),f(α₂), . . . f(α_(n))): f∈

[x],deg(f)≤k−1},

where deg( )denotes the degree of a polynomial, f(x)=u₀+u₁x+u₂x²+ . . .+u_(k−1)x^(k−1), and, u_(i)∈

, i=0, 1, . . . , k

1 are the messages. Every evaluation symbol f(α), α∈A, is called a codeword symbol or a storage node. The sub-packetization size is defined asr, and r

n−k denotes the number of parity symbols.

Assume e nodes fail, e≤n

k, and we want to recover them. The number of helper nodes are denotedby d. The amount of information transmitted from the helper nodes isdefined as the repair bandwidth b, measured in the number of symbolsover GF(q). All the remaining n

e=d nodes are assumed to be the helper nodes unless stated otherwise. Wedefine the normalized repair bandwidth as

$\frac{b}{\; d},$

which is the average fraction of information transmitted from each theminimum storage regenerating (MSR) bound for the bandwidth is

$\begin{matrix}{b \geq {\frac{e\; \; d}{d - k + e}.}} & (2)\end{matrix}$

As mentioned before, codes achieving the MSR bound require largesub-packetization sizes. In this section, we focus on the single erasurecase.

Assume B≤F, namely, B is a subfield of F. A linear repair schemerequires some symbols of the subfield B to be transmitted from eachhelper node. If the symbols from the same helper node are linearlydependent, the repair bandwidth decreases. In particular, the schemeuses dual code to compute the failed node and uses trace function toobtain the transmitted subfield symbols, as detailed below. Assume f(α*)fails for some α*∈A. For any polynomial p(x)∈F[x] of which the degree issmaller than r, (ν1p(α1), ν2p(α2), . . . , νnp(αn)) is a dual codewordof RS (A, k), where νi, i∈[n] are non-zero constants determined by theset A. We can thus repair the failed node f(α*) from

$\begin{matrix}{{\upsilon_{\alpha^{*}}{p\left( \alpha^{*} \right)}{f\left( \alpha^{*} \right)}} = {- {\overset{n}{\sum\limits_{{i = 1},{\alpha_{i} \neq \alpha^{*}}}}{v_{i}{p\left( \alpha_{i} \right)}{f\left( \alpha_{i} \right)}}}}} & (3)\end{matrix}$

The summation on the right side means that we add all the

i elements from i=1 to i=n except when α₁≠α*.

The trace function from F onto B is defined as

(β)=β+β^(q)+ . . . +

,  (4)

where β∈F, B=GF(q) is called the base field, and q is a power of a primenumber. It is a linear mapping from F to B and satisfies

(αβ)=

(β)  (5)

for all α∈B.

We define the rank rank_(B)({γ1, γ2, . . . , γi}) to be the cardinalityof a maximal subset of {γ1, γ2, . . . , γi} that is linearly independentover B. For example, for B=GF(2) and α ∈/B,rankB({1, α, 1+α})=2 becausethe subset {1, α} is the maximal subset that is linearly independentover B and the cardinality of the subset is 2.

Assume we use polynomials p_(j)(x), j∈[

] to generate different dual codewords, called repair polynomials.Combining the trace function and the dual code, we have

$\begin{matrix}{{{tr}_{/}\left( {v_{\alpha^{*}}{p_{j}\left( \alpha^{*} \right)}{f\left( \alpha^{*} \right)}} \right)} = {- {\sum\limits_{{i = 1},{\alpha_{i} \neq \alpha^{*}}}^{n}{{{tr}_{/}\left( {\upsilon_{i}{p_{j}\left( \alpha_{i} \right)}{f\left( \alpha_{i} \right)}} \right)}.}}}} & (6)\end{matrix}$

In a repair gee e helper f(αi)transmits

{

(ν_(i) p _(j)(α_(i))f(α_(i))):j∈[

]}.  (7)

TABLE 1 shows a comparison of different schemes for single erasure. Whena=l, the scheme is one coset. When a=1, the scheme in multiple cosets.

TABLE I repair bandwith code length restrictions Schemes in [6], (n −1( 

 −

 ) n ≤  

q 

 ≤ r [7] Scheme in [16] $< {\frac{}{r}\left( {n + 1} \right)}$ n =log,  

Scheme in [17] $\frac{\text{?}}{r}\left( {n - 1} \right)$ n^(n) ≈  

Our scheme in one coset$\leq {\frac{}{r}\left( {n - 1} \right)\left( {a - s} \right)}$ n ≤(q^(a) − 1) q 

 ≤ r, a| 

Our scheme in two cosets $< {\left( {n - 1} \right)\frac{\text{?}}{2}}$n ≤ 2(q^(a) − 1) ${\frac{}{r} \leq a},{a}$ Our scheme in multiplecosets 1$< {\frac{}{r}\left( {n + 1 + {\left( {r - 1} \right)\left( {q^{n} - 2} \right)}} \right)}$n ≤ (q^(a) − 1)m

 |a = r^(m) for some interger m Our scheme in multiple cosets 2$\frac{}{r}\left( {n - 1 + {\left( {r - 1} \right)\left( {q^{n} - 2} \right)}} \right)$n ≤ (q^(a) − 1)m

 |a ≈ m^(m) for some interger m

indicates data missing or illegible when filed

TABLE II repair bandwith code length restrictions Scheme 1 in [19]$\leq {{\left( {n - e} \right)e} - \frac{\text{?}}{2}}$ n ≤ q 

  ${{q\; \text{?}} \leq r},\; {\text{?} < \sqrt{\log_{q},n}}$ Scheme2 in [19]$\leq {\min\limits_{\text{?}}\left( {\left( {n - \text{?}}\; \right)\left( { - \left\lfloor {\log_{q}\left( \frac{\text{?}}{{2\mspace{11mu} \text{?}} - 1} \right)} \right\rfloor} \right)} \right)}$n ≤ q 

  Scheme in [22] $\frac{\text{?}}{d - \text{?}}$ n^(n) ≈  

Our scheme for multiple erasures in one coset ≤ ?  ( n - e )  ( a - s) n ≤ (q^(a) − 1)${{q} \leq r},{a},{e < {\frac{1}{a - s}\sqrt{\log_{q},n}}}$ Ourscheme for multiple erasures in multiple coset n - k  ( n - e + ( n -k + e )  ( q   - 2 ) ) n ≤ (q^(a) − 1)m

 |a ≈ m^(m) for some interger m

indicates data missing or illegible when filed

TABLE II shows a comparison for multiple erasures. When a=l and s=l, thescheme is one coset. When a=l, the scheme in multiple cosets.

Supposing {ν_(α)*p₁(α*),ν_(α)*p₂(α*), . . . ν_(α)*

(α*)} is a basis for F over B, and assume {μ1, μ2, . . . , μ′} is itsdual basis. Then, f(α*) can be repaired by

$\begin{matrix}{{f\left( \alpha^{*} \right)} = {\sum\limits_{j = 1}^{}{\mu_{j}{{{tr}_{/}\left( {\upsilon_{\alpha^{*}}{p_{j}\left( \alpha^{*} \right)}{f\left( \alpha^{*} \right)}} \right)}.}}}} & (8)\end{matrix}$

Since νa*is a non-zero constant, we equivalently suppose that {p1(α*), .. . , p′(α*)} is a basis.

In fact, by any linear repair scheme of RS code for the failed nodef(α*) is equivalent to choosing p_(j)(x), j∈[

], with degree smaller than r, such that {p₁(α*), . . .

(α*)} forms a basis for F over B. We call this the full rank condition:

({p ₁(α*),p ₂(α*), . . . ,

(α*)})=

,  (9)

The repair bandwidth can be calculated from (7) and by noting thatv_(i)f(α_(i)) is a constant:

$\begin{matrix}{b = {\sum\limits_{{\alpha \in A},{\alpha \neq \alpha^{*}}}{{{rank}_{}\left( \left\{ {{p_{1}(\alpha)},{p_{2}(\alpha)},\ldots \mspace{14mu},{p_{}(\alpha)}} \right\} \right)}.}}} & (10)\end{matrix}$

We call this the repair bandwidth condition.

The goal of a good RS code construction and its repair scheme is tochoose appropriate evaluation points A and polynomials p_(j)(x), j∈[

], that can reduce the repair bandwidth in (10)while satisfying (9).

The following lemma is due to the structure of the multiplicative groupof F, which will be used for finding text use

-   -   Lemma 1. Assume        ≤        =GF(        ), then        can be partitioned to

$t\overset{\Delta}{=}\frac{q^{} - 1}{{} - 1}$

cosets: {

,β

*, β²

*, . . . ,β^(t−1)

*}, where β is a primitive element of

.

-   -   Proof: The        −1 elements in        * are {1,β,β², . . .        ⁻²} and        * ⊆        * Assume that t is the smallest nonzero number that satisfies        β^(t)∈        *, then we know that β^(k)∈        *if and only if t|k. Also, β^(k) ¹ ≠β^(k) ² when k₁≠k₂ and        k₁,k₂<        −2. Since there are only |        |−1 nonzero distinct elements in        and        =1, we have

$t = \frac{q^{} - 1}{{{} - 1}\;}$

and the t cosets are

*{1,β^(t),β^(2t), . . . ,

}, β

*={β,β^(t+1),β^(2t+1), . . . ,

, . . . β^(t−1)

*={β^(t−1),β^(2t−1),β^(3t+1), . . . ,

}.

Reed-Solomon Repair Schemes for Single Erasure.

Three RS repair schemes for single erasure are provided. Evaluationpoints are part of one coset, two cosets and multiple cosets for asingle erasure. From these constructions, embodiments can achievedifferent points on the tradeoff between the sub-packetization size andnormalized repair bandwidth.

Processes may be based on schemes at employ:

i) taking an original RS code and constructing a new code over a largerfinite field—thus the sub-packetization of l is increased,ii) for the schemes using one and two cosets, the code parameters n, kare kept the same as the original code. Hence, for given n, r=n

k, the sub-packetization size l increases, but we show that thenormalized repair bandwidth remains the same, andiii) For the scheme using multiple cosets, the code length n isincreased and the redundancy r is fixed. Moreover, the code length ngrows faster than the sub-packetization size l. Therefore, for fixed n,r, the sub-packetization 1 decreases, and we show that the normalizedrepair bandwidth is only slightly larger than the original code.

A. Schemes in One Coset

Assume E=GF(q^(a)) is a subfield of F=GF(q^(l)) and B=GF(q) is the basefield, where q is a prime number. The evaluation points of the code overF that we construct are part of one coset in Lemma 1.

We first present the following lemma about the basis.

Lemma 2. Assume {ξ₁,ξ₂, . . .

} is a basis for

=GF(

) over

=GF(q), then {ξ₁ ^(q) ^(s) ,ξ₂ ^(q) ^(s) , . . . ,

}, s∈[

] is also a basis.

Proof: Assume {ξ₁ ^(q) ^(s) ,ξ₂ ^(q) ^(s) , . . . ,

}, s∈[

] is not a basis for

over

, then there exist nonzero (α₁,α₂, . . . ,

), α_(i)∈

, i∈[

], that satisfy

$\begin{matrix}{{{{\alpha_{1}\xi_{1}^{q^{s}}} + {\alpha_{2}\xi_{2}^{q^{s}}} + \ldots + {\alpha_{}\xi_{}^{q^{s}}}} = {0 = \left( {{\alpha_{1}\xi_{1}} + {\alpha_{2}\xi_{2}} + \ldots + {\alpha_{}\xi_{}}} \right)^{q^{s}}}},} & (11)\end{matrix}$

which is in contradiction to the assumption that {ξ₁,ξ₂, . . .

} is a basis for

over

.

The following theorem shows the repair scheme using one coset for theevaluation points.

Theorem 1. There exists an RS(n,k) code over

=GF(

) with repair bandwidth

$b \leq {\frac{}{a}\left( {n - 1} \right)\left( {a - s} \right)}$

symbols over

=GF(q), where q is a prime number and a, s satisfy n<q^(a),q^(s)≤n−k, a|

.

Proof: Assume a field

=GF(

) is extended from

=GF(q^(a)), a|

, and β is a primitive element of

. We focus on the code RS(A, k) of dimension k over

with evaluation points A={α₁,α₂, . . . , α_(n)}β^(m)

* for some

${0 \leq m < \frac{q^{} - 1}{q^{\alpha} - 1}},$

which is one of the cosets in Lemma 1.

The base field is B=GF(q) and (6) is used to repair the failed nodef(α*).

Construction I: For s=a

1, we choose

$\begin{matrix}{{{p_{j}(x)} = \frac{{tr}_{/}\left( {\xi_{j}\left( {\frac{x}{\beta^{m}} - \frac{\alpha^{*}}{\beta^{m}}} \right)} \right)}{\frac{x}{\beta^{m}} - \frac{\alpha^{*}}{\beta^{m}}}},{j \in \lbrack a\rbrack},} & (12)\end{matrix}$

Where {ξ1, ξ2, . . . , ξa} is a basis for E over B. The degree of pj(x)is smaller than r since q^(s)≤r. When x=α*, by (4) we have

p _(j)(α*)=ξ_(j).  (13)

So the polynomials satisfy

({p ₁(α*),p ₂(α*), . . . ,p _(a)(α*)})=a.  (14)

When x≠α*, since

${{{tr}_{/}\left( {\xi_{j}\left( {\frac{x}{\beta^{m}} - \frac{\alpha^{*}}{\beta^{m}}} \right)} \right)} \in },{{{and}\mspace{14mu} \frac{x}{\beta^{m}}} - \frac{\alpha^{*}}{\beta^{m}}}$

is a constant independent of j, we have

({p ₁(x),p ₂(x), . . . ,p _(a)(x)})=1.  (15)

Let {η₁,η₂,η₃, . . . ,

} be a basis for

over

the

repair polynomials are chosen as

{η₁ p _(j)(x),η₂ p _(j)(x), . . .

p _(j)(x): j∈[a]}  (16)

Since p_(j)(x)∈

, we can conclude that

$\begin{matrix}{{{rank}_{}\left( \left\{ {{\eta_{1}{p_{j}\left( \alpha^{*} \right)}},{\eta_{2}{p_{j}\left( \alpha^{*} \right)}},\ldots \mspace{14mu},{{\eta_{/a}{p_{j}\left( \alpha^{*} \right)}\text{:}j} \in \lbrack a\rbrack}} \right\} \right)} = {{\frac{}{a}{{rank}_{}\left( \left\{ {{p_{1}\left( \alpha^{*} \right)},{p_{2}\left( \alpha^{*} \right)},\ldots \mspace{14mu},{p_{a}\left( \alpha^{*} \right)}} \right\} \right)}} = }} & (17)\end{matrix}$

satisfies the full rank condition and for x≠α*

$\begin{matrix}{{{rank}_{}\left( \left\{ {{\eta_{1}{p_{j}(x)}},{\eta_{2}{p_{j}(x)}},\ldots \mspace{14mu},{{\eta_{/a}{p_{j}(x)}\text{:}j} \in \lbrack a\rbrack}} \right\} \right)} = {{\frac{}{a}{{rank}_{}\left( \left\{ {{p_{1}(x)},{p_{2}(x)},\ldots \mspace{14mu},{p_{a}(x)}} \right\} \right)}} = {\frac{}{a}.}}} & (18)\end{matrix}$

From (10) we can calculate the repair bandwidth

$\begin{matrix}{b = {\frac{}{a}{\left( {n - 1} \right).}}} & (19)\end{matrix}$

Construction II: For s≤a

1,

$\begin{matrix}{{{p_{j}(x)} = {\xi_{j}{\prod\limits_{i = 1}^{q^{*} - 1}\left( {\frac{x}{\beta^{m}} - \left( {\frac{\alpha^{*}}{\beta^{m}} - {w_{i}^{- 1}\xi_{j}}} \right)} \right)}}},{j \in \lbrack a\rbrack},} & (20)\end{matrix}$

where {ξ₁,ξ₂, . . . , ξ_(a)} is a basis for

over

and W={w₀=0,w₁,w₂, . . . , w_(q) _(s) ⁻¹} is an s-dimensional subspacein

, s<a, q^(s)≤r. It is easy to check that the degree of p_(j)(x) issmaller than r since q^(s)≤r. When x=α*, we have

$\begin{matrix}{{p_{j}\left( \alpha^{*} \right)} = {\xi_{j}^{q^{s}}{\prod\limits_{i = 1}^{q^{s} - 1}{w_{i}^{- 1}.}}}} & (21)\end{matrix}$

Since

$\prod\limits_{i = 1}^{q^{s} - 1}w_{i}^{- 1}$

is a constant, from Lemma 2 we have

({p ₁(α*),p ₂(α*), . . . ,p _(a)(α*)})=α.  (22)

For x≠α*, set

${x^{\prime} = {{\frac{\alpha^{*}}{\beta^{m}} - \frac{x}{\beta^{m}}} \in }},$

we have

$\begin{matrix}\begin{matrix}{{p_{j}(x)} = {\xi_{j}{\prod\limits_{i = 1}^{q^{s} - 1}\left( {\frac{x}{\beta^{m}} - \left( {\frac{\alpha^{*}}{\beta^{m}} - {w_{i}^{- 1}\xi_{j}}} \right)} \right)}}} \\{= {\xi_{j}{\prod\limits_{i = 1}^{q^{s} - 1}\left( {{w_{i}^{- 1}\xi_{j}} - x^{\prime}} \right)}}} \\{= {\xi_{j}{\prod\limits_{i = 1}^{q^{s} - 1}{\left( {w_{i}^{- 1}x^{\prime}} \right){\prod\limits_{i = 1}^{q^{s} - 1}\left( {{\xi_{j}/x^{\prime}} - w_{i}} \right)}}}}} \\{= {\left( x^{\prime} \right)^{q^{s}}{\prod\limits_{i = 1}^{q^{s} - 1}{\left( w_{i}^{- 1} \right){\prod\limits_{i = 0}^{q^{s} - 1}{\left( {{\xi_{j}/x^{\prime}} - w_{i}} \right).}}}}}}\end{matrix} & (23)\end{matrix}$

Then,

${{g(y)} = {\prod\limits_{i = 0}^{q^{s} - 1}\left( {y - w_{i}} \right)}},$

is a linear mapping from E to itself with dimension a

s over B. Since

$\left( x^{\prime} \right)^{q^{s}}{\prod\limits_{i = 1}^{q^{s} - 1}\left( w_{i}^{- 1} \right)}$

is a constant independent of j, we have

({p ₁(x)p ₂(x), . . . p _(a)(x)})≤a−s.  (24)

Let {η₁,η₂,η₃, . . . ,

} be a basis for

over

, then the

polynomals are chosen as {η₁p_(j)(x), η₂p_(j)(x), . . . ,η_(l/a)p_(j)(x), j∈[a]}. From (21) and (23) we know that p_(j)(x)∈

so e can conclude that

$\begin{matrix}{{{rank}_{}\left( \left\{ {{\eta_{1}{p_{j}\left( \alpha^{*} \right)}},{\eta_{2}{p_{j}\left( \alpha^{*} \right)}},\ldots \mspace{14mu},{{\eta_{/a}{p_{j}\left( \alpha^{*} \right)}\text{:}j} \in \lbrack a\rbrack}} \right\} \right)} = {{\frac{}{a}{{rank}_{}\left( \left\{ {{p_{1}\left( \alpha^{*} \right)},{p_{2}\left( \alpha^{*} \right)},\ldots \mspace{14mu},{p_{a}\left( \alpha^{*} \right)}} \right\} \right)}} = }} & (25)\end{matrix}$

satisfies (9), and for x≠α*

$\begin{matrix}{{{rank}_{}\left( \left\{ {{\eta_{1}{p_{j}(x)}},{\eta_{2}{p_{j}(x)}},\ldots \mspace{14mu},{{\eta_{/a}{p_{j}(x)}\text{:}j} \in \lbrack a\rbrack}} \right\} \right)} = {{\frac{}{a}{{rank}_{}\left( \left\{ {{p_{1}(x)},{p_{2}(x)},\ldots \mspace{14mu},{p_{a}(x)}} \right\} \right)}} \leq {\frac{}{a}{\left( {a - s} \right).}}}} & (26)\end{matrix}$

Now from (10) we can calculate the repair bandwidth

$\begin{matrix}{b \leq {\frac{}{a}\left( {n - 1} \right){\left( {a - s} \right).}}} & (27)\end{matrix}$

Rather than directly using existing schemes, the polynomials (12) and(20) uses a set of basis {ξ₁, ξ₂, . . . , ξ_(a)} from E to B. Moreover,each polynomial is multiplied with the basis for F over E to satisfy thefull rank condition. In this case, our embodiments and schemessignificantly reduce the repair bandwidth when the code length remainsthe same. Our evaluation points are in a coset rather than the entirefield F. It should be noted that a here can be an arbitrary number thatdivides l and when a=l. Note that the normalized repair bandwidth

$\frac{b}{\left( {n - 1} \right)}$

decreases as a decreases. Therefore, our scheme outperforms existingschemes when applied to the case of

>log_(q).

Example 1. Assume q=2, l=9, a=3 and E={0, 1, α,α2, . . . , α6}. LetA=E*, n=7, k=5 so r=n

k=2. Choose s=log₂ r=1 and W={0, 1} in Construction IL Then, we havepj(x)=& ξj(x

α*+ξj). Let {ξ1, ξ2, ξ3} be {1, α,α2}. It is easy to check thatrankB({p1(α*), p2(α*), p3(α*)})=3 and rankB({p1(x), p2(x), p3(x)})=2 forx=6 α*. Therefore the repair bandwidth is b=36 bits as suggested inTheorem 1. For the same (n, k, l), the repair bandwidth in prior schemesmay be 48 bits. For another example, consider RS (14, 10) code used inFacebook, we have repair bandwidth of 52 bits for l=8, while the priorscheme requires 60 bits and the naive scheme requires 80 bits.

According to one embodiment, the scheme constructs a scalar code. Thisscalar code may be the first example of such a scalar code in the art.

B. Schemes in Two Cosets

According to one embodiment, the scheme can include evaluation pointschosen from two cosets. In this scheme, polynomials are chosen that havefull rank when evaluated at the coset containing the failed node, andrank 1 when evaluated at the other coset.

Theorem 2. There exists an RS(n,k) code over

=GF (

) with repair bandwidth

$b < {\left( {n - 1} \right)\frac{ + a}{2}}$

symbols over

=GF(q), where q is a prime number and a satisfies

${n \leq {2\left( {q^{a} - 1} \right)}},\left. a \middle|  \right.,{\frac{}{a} \leq {n - {k.}}}$

Proof: Assume a field

=GF(

) is extended from

=GF(q^(a)) and β is the primitive element of

. We focus on the code RS(A,k) over

of dimension k with evaluation points A consisting of n/2 ponts fromβ^(m) ¹

* and n/2 points from β^(m) ²

*,

${{0 \leq m_{1} < m_{2} \leq {{\frac{q^{} - 1}{q^{a} - 1}\mspace{14mu} {and}\mspace{14mu} m_{2}} - m_{1}}} = q^{s}},{s \in {\left\{ {0,1,\ldots \mspace{14mu},\frac{}{a}} \right\}.}}$

In this case we view

as the base field and repair the failed node f(α*) by

$\begin{matrix}{{{tr}_{/}\left( {\upsilon_{\alpha^{*}}{p_{j}\left( \alpha^{*} \right)}{f\left( \alpha^{*} \right)}} \right)} = {- {\overset{n}{\sum\limits_{{i = 1},{\alpha_{i} \neq \alpha^{*}}}}{{{tr}_{/}\left( {\upsilon_{i}{p_{j}\left( \alpha_{i} \right)}{f\left( \alpha_{i} \right)}} \right)}.}}}} & (28)\end{matrix}$

Inspired by [6, Theorem 10], for

${j \in \left\lbrack \frac{}{a} \right\rbrack},$

we choose

$\begin{matrix}{{p_{j}(x)} = \left\{ \begin{matrix}{\left( \frac{x}{\beta^{w_{2}}} \right)^{j - 1},} & {{{{if}\mspace{14mu} \alpha^{*}} \in {\beta^{m_{1}}^{*}}},} \\{\left( \frac{x}{\beta^{w_{1}}} \right)^{j - 1},} & {{{{if}\mspace{14mu} \alpha^{*}} \in {\beta^{m_{2}}^{*}}},}\end{matrix} \right.} & (29)\end{matrix}$

The degree of p_(j)(x) is smaller than r when

$\frac{}{a} \leq {r.}$

Then,

we check the rank in each case.

When α*∈β^(m) ¹

*, if x=β^(m) ¹ γ∈β^(m) ¹

*, for some γ∈

*,

$\begin{matrix}{{{p_{j}(x)} = {\left( \frac{x}{\beta^{m_{1}}} \right)^{j - 1} = \gamma^{i - 1}}},} & (30)\end{matrix}$

so

$\begin{matrix}{{{rank}_{}\left( \left\{ {{p_{1}(x)},{p_{2}(x)},\ldots \;,{p_{\frac{}{a}}(x)}} \right\} \right)} = 1.} & (31)\end{matrix}$

If β^(m) ² γ∈β^(m) ²

*, for some γ∈

*,

$\begin{matrix}{{p_{j}(x)} = {\left( \frac{x}{\beta^{m_{1}}} \right)^{j - 1} = {\left( \beta^{m_{2} - m_{1}} \right)^{j - 1}{\gamma^{j - 1}.}}}} & (32)\end{matrix}$

Since m₂−m₁=q^(s) and

$\left\{ {1,\beta,\beta^{2},\ldots \;,\beta^{\frac{}{a} - 1}} \right\}$

is the polynomial basis for

over

, from Lemma 2 we know that

$\begin{matrix}{{{rank}_{}\left( \left\{ {{p_{1}(x)},{p_{2}(x)},\ldots \;,{p_{\frac{}{a}}(x)}} \right\} \right)} = {\frac{}{a}.}} & (33)\end{matrix}$

When α*∈β^(m) ¹

*, if x=β^(m) ¹ γ∈β^(m) ¹

* for some γ∈

*,

$\begin{matrix}\begin{matrix}{{p_{j}(x)} = \left( \frac{x}{\beta^{m_{2}}} \right)^{j - 1}} \\{= {\left( \beta^{m_{1} - m_{2}} \right)^{j - 1}\gamma^{j - 1}}} \\{= {\left( \beta^{m_{1} - m_{2}} \right)^{1 - \frac{}{a}}\left( \beta^{m_{2} - m_{1}} \right)^{\frac{}{a} - j}{\gamma^{j - 1}.}}}\end{matrix} & (34)\end{matrix}$

Since

$\left( \beta^{m_{2} - m_{1}} \right)^{1 - \frac{}{a}}$

is a constant, from Lemma 2 we know that

$\begin{matrix}{{{rank}_{}\left( \left\{ {{p_{1}(x)},{p_{2}(x)},\ldots \;,{p_{\frac{}{a}}(x)}} \right\} \right)} = {\frac{}{a}.}} & (35)\end{matrix}$

If x=β^(m) ² γ∈β^(m) ²

* for some γ∈

*,

$\begin{matrix}{{{p_{j}(x)} = {\left( \frac{x}{\beta^{m_{2}}} \right)^{j - 1} = \gamma^{j - 1}}},} & (36)\end{matrix}$

so

$\begin{matrix}{{{rank}_{}\left( \left\{ {{p_{1}(x)},{p_{2}(x)},\ldots \;,{p_{\frac{}{a}}(x)}} \right\} \right)} = 1.} & (37)\end{matrix}$

Therefore,

$\left\{ {{p_{j}\left( \alpha^{*} \right)},{j \in \left\lbrack \frac{}{a} \right\rbrack}} \right\}$

has full rank over E, for any evaluation point α* ∈A. For x from thecoset containing α*, the polynomials have rank a, and for x from theother coset, the polynomials have rank 1. Then, the repair bandwidth insymbols over B can be calculated from (10) as

$\begin{matrix}\begin{matrix}{b = {{\frac{}{a}\left( {\frac{n}{2} - 1} \right)\log_{q}{}} + {\frac{n}{2}\log_{q}{}}}} \\{= {{\left( {n - 1} \right)\frac{ + a}{2}} - \frac{ - a}{2}}} \\{< {\left( {n - 1} \right){\frac{ + a}{2}.}}}\end{matrix} & (38)\end{matrix}$

Example 2. Take the RS (14, 11) code over F=GF(2¹²) for example. Let βbe the primitive element in F, a=4, s=l/a=3 and A=E*∪βE*. Assume α*∈βE*,then {pj(x), j∈[3]} is the set {1, x, x²}. It is easy to check that whenx∈βE* the polynomials have full rank and when x∈E*the polynomials haverank 1. The total repair bandwidth is 100 bits. For the same (n, k, l),the repair bandwidth of our scheme in one coset is 117 bits. For priorschemes, which only works for l/a=2, we can only choose a=6 and get therepair bandwidth of 114 bits for the same (n, k, l).

C. Schemes in Multiple Cosets

In the schemes in this subsection, we extend an original code to a newcode over a larger field and the evaluation points are chosen frommultiple cosets in Lemma 1 to increase the code length. The constructionensures that for fixed n, the sub-packetization size is smaller than theoriginal code. If the original code satisfies several conditions to bediscussed soon, the repair bandwidth in the new code is only slightlylarger than that of the original code.

Particularly, if the original code is an MSR code, then we can get thenew code in a much smaller sub-packetization level with a small extrarepair bandwidth. Also, if the original code works for any number ofhelpers and multiple erasures, the new code works for any number ofhelpers and multiple erasures, too. We discuss multiple erasures below.

We first prove a lemma regarding the ranks over different base fields,and then describe the new code.

-   -   Lemma 1. Let        =GF(q),        ′=GF(        ),        =GF(q^(a)),        =GF(        ),        =a        ′. a and        ′ are relatively prime and q can be any power of a prime number.        For any set of {γ₁,γ₂, . . . , γ_(l′)}⊆        ′≤        , we have

({γ₁,γ₂, . . . ,

})

({γ₁,γ₂, . . . ,

})  (39)

Proof: Assume ranks

({γ₁,γ₂, . . . ,

})=c and without loss of generality, {γ₁,γ₂, . . . , γ_(c)} are linearlyin n-dent over

. Then, we can construct {γ_(c+1)′,γ_(c+2), . . . ,

}⊆

′ to make {γ₁,γ₂, . . . , γ_(c), γ_(c+1)′,γ_(c+2)′, . . . ,

} form a basis for

over

.

Assume we get F by adjoining β to B. Then, {1, β,β2, . . . , β′

1} is a basis for both F over E, and F over B. So, any symbol y∈F can bepresented as a linear combination of {1,β,β2, . . . , β′

1} with some coefficients in E. Also, there is an invertible lineartransformation with coefficients in B between {γ1,γ2, . . . , γc, γ0c+1,γ0c+2, . . . ,γ0′

1} and {1,β,β2, . . . , β′

1}, because they are a basis for F0 over B. Combined with the fact that{1,β,β2, . . . , β′

1} is also a basis for F over E, we can conclude that any symbol y∈F canbe represented as

y=x ₁γ₁ +x ₂γ₂ + . . . +x _(c)γ_(c) +x _(c+1)γ_(c+1)′+ . . . +

  (40)

with some coefficients xi∈E, which means that {γ₁,γ₂, . . .,γ_(c),γ_(0c+1),γ_(0c+2), . . . , γ₀′} is also a basis for F over E.Then, we have that {γ₌₁,γ₂, . . . , γ_(c)} are linearly independent overE,

({γ₁,γ₂, . . . ,

})≥=

({γ₁,γ₂, . . . ,

})  (41)

Since

≤

, we also have

({γ₁,γ₂, . . . ,

})=

({γ₁,γ₂, . . . ,

})  (42)

Theorem 3. Assume there exists a RS(n′,k′) code

over

′=GF(

) with evaluation points set A^(l). The evaluation points are linearlyindependent over B=GF(q). The repair bandwidth is b and the repairpolynomials are p_(j)′(x).

Then, we can construct a new RS(n, k) code E over

=GF(

),

=a

with n=(q^(a)−1)n′, k=n−n′+k′ and repair bandwidth ofb=ab′(q^(a)−1)+(q^(a)−2)

; symbols over B=GF(q) if we can find new repair polynomials p_(j)(x)∈

[x],j∈[

], with degrees less than n H k that satisfy

({p ₁(x),p ₂(x), . . . ,

(x)})=

({p ₁′(α),p ₂′(α), . . . ,

(α)})  (43)

Proof: We first prove the case when a and

are necessarily relatively prime using Lemma 3, the case when a and

′ are not relatively prime are proved in Appendix A. Assume theevaluation points of

′ are A′={α₁,α₂, . . . , α_(n′)}, then from Lemma 3 we know that theyare also linearly independent over

, so there does not exist γ_(i),γ_(j)∈

* that satisfy α_(i)γ_(i)=α_(j)γ_(k), which implies that {α₁

*, α₂

*, . . . , α_(n′)

*} are distinct cosets. Then, we can extend the evaluation points to be

A={α ₁

*,α₂

*, . . . ,α_(n′)

}  (44)

and n=(q^(a)−1)n′. We keep the same redundancy r=n′−k′ for the new codek=n−r.

For the new code

we use p_(j)(x)∈

[x], j∈[

] to repair the failed node f(α*)

$\begin{matrix}{{{tr}_{/}\left( {\upsilon_{\alpha}*{p_{j}\left( \alpha^{*} \right)}{f\left( \alpha^{*} \right)}} \right)} = {- {\sum\limits_{{\alpha \in A},{\alpha \neq \alpha^{*}}}{{{tr}_{/}\left( {\upsilon_{\alpha}{p_{j}(\alpha)}{f(\alpha)}} \right)}.}}}} & (45)\end{matrix}$

Assume the failed node is f(α*) and α*∈α_(i)

*. Then, for the node x∈α_(i)

*, because the original code satisfies the full rank condition, we have

({p ₁(x),p ₂(x), . . . ,

(x)})=

({p ₁′(α_(i)),p ₂′(α_(i)), . . . ,

(α_(i))})=

  (43)

then we can recover the failed node with p_(j)(x), and each helper inthe coset containing the failed node transmits

′ symbols over

.

For helper in the other cosets, x∈α_(ϵ)

*, ϵ≠i, by (43),

({p ₁(x),p ₂(x), . . . ,

(x)})=

({p ₁′(α_(ϵ)),p ₂′(α_(ϵ)), . . . ,

(α_(ϵ))})=

  (47)

then evey helper in these cosets transmits

$\frac{b^{\prime}}{n^{\prime} - 1}$

symbols in

on average.

The repair bandwidth of the new code can be calculated from the repairbandwidth condition (10) as

$\begin{matrix}\begin{matrix}{b = {{{\frac{b^{\prime}}{n^{\prime} - 1} \cdot \left( {n^{\prime} - 1} \right)}{{^{*}} \cdot a}} + {\left( {{^{*}} - 1} \right){^{\prime} \cdot a}}}} \\{= {{{ab}^{\prime}\left( {q^{a} - 1} \right)} + {\left( {q^{a} - 2} \right)}}}\end{matrix} & (48)\end{matrix}$

Note that the calculation in (48) and (38) are similar in the sense thata helper in the coset containing the failure naively transmits theentire stored information, and the other helpers use the bandwidth thatis the same as the original code. As a special case of Theorem 3,

$b^{\prime} = {\frac{^{\prime}}{r}\left( {n^{\prime} - 1} \right)}$

matching the MSR bound (1), we get

$\begin{matrix}{{b = {{\frac{}{r}\left( {n - 1} \right)} + {\frac{}{r}\left( {r - 1} \right)\left( {q^{a} - 2} \right)}}},} & (49)\end{matrix}$

where the second term is the extra bandwidth compared to the MSR bound.

Next, we apply Theorem 3 to the near-MSR code and the MSR code.

Theorem 4. There exists an RS(n,k) code over

=GF(

) of which

$n = {\left( {q^{a} - 1} \right)\log_{r}\frac{}{a}}$

and a|

, such that the repair bandwidth satisfies

${b < {\frac{}{n - k}\left\lbrack {n + 1 + {\left( {n - k - 1} \right)\left( {q^{q} - 2} \right)}} \right\rbrack}},$

measured in symbols over

=GF(q) for some prime number q.

Proof: We first prove the case when a and

are relatively prime using Lemma 3, the case when a and

′ are not necessarily relatively prime are proved in Appendix A. We usethe code in [16] as the or original code. The original code is definedin

′=GF (

) and

=r^(n)′. The evaluation points are

A^(′) = {β, β^(r), B^(r²), …  , B^(r^(n^(′) − 1))}

where β is a primitive element of

′.

In the original code, for c=0,1, 2, . . . ,

−1, we write its r-ary expansion as c=(c_(n′),c_(n′−1) . . . c₁), where0≤c_(i)≤r−1 is the i-th digit from the right. Assuming the failed nodeis f(β^(r) ^(i−1) ), the repair polynomials are chosen to be

p _(j)′(x)=β^(c) x ^(s) ,c _(i)=0,s=0,1,2, . . . ,r−1,x∈

′.  (50)

Here e varies from 0 to

−1 given that c=0, and s varies from 0 to r−1. So, we have polynomialsin total. The subscript j is indexed by c and s, and by a small abuse ofthe notation, we write j∈[

].

In the new code, let us define

=GF(q^(a)) of which a and

are relatively prime. Adjoining β to

, we get

=GF(

),

≤a

. The new evaluation points are

A = {β ^(*), β^(r) ^(*), B^(r²)^(*), …  , B^(r^(n^(′) − 1))^(*)}.

Since A′ is part of the polynomial basis for

over

know that

{β, β^(r), B^(r²), …  , B^(r^(n^(′) − 1))}

are linearly independent over

, Hence, we can apply Lemma 3 and the cosets are distinct resulting in

$n = {{A} = {\left( {q^{\alpha} - 1} \right)\log_{r}{\frac{}{a}.}}}$

In our new code, let us assume the failed node is f(α*) and α*∈β^(r)^(i−1) , and we choose the polynomial p_(j)(x) with the same form asp_(j)′(x),

p _(j)(x)=β^(c) x ^(s) ,c _(i)=0,s=0,1,2, . . . ,r−1,x∈

  (51)

For nodes corresponding to x=β^(r) ^(i) γ∈

* , for some γ∈

*, we know that

p _(j)(x)=β^(c) x ^(s)=β^(c)(γβ^(r) ^(t) )^(s)=γ^(s) p _(j)′(β^(r) ^(t))  (52)

Since p_(j)′(β^(r) ^(t) )∈

′, from Lemma 3, we have

$\begin{matrix}{{{{rank}_{}\left( \left\{ {{\gamma^{s}{p_{1}^{\prime}\left( \beta^{r^{t}} \right)}},{\gamma^{s}{p_{2}^{\prime}\left( \beta^{r^{t}} \right)}},\ldots \mspace{14mu},{\gamma^{s}{p_{^{\prime}}^{\prime}\left( \beta^{r^{t}} \right)}}} \right\} \right)} = {{{rank}_{}\left( \left\{ {{p_{1}^{\prime}\left( \beta^{r^{t}} \right)},{p_{2}^{\prime}\left( \beta^{r^{t}} \right)},\ldots \mspace{14mu},{p_{^{\prime}}^{\prime}\left( \beta^{r^{t}} \right)}} \right\} \right)} = {{rank}_{}\left( \left\{ {{p_{1}^{\prime}\left( \beta^{r^{t}} \right)},{p_{2}^{\prime}\left( \beta^{r^{t}} \right)},\ldots \mspace{14mu},{p_{^{\prime}}^{\prime}\left( \beta^{r^{t}} \right)}} \right\} \right)}}},} & (53)\end{matrix}$

which satisfies (43). Since repair bandwidth of the original code is

${b^{\prime} < {\left( {n^{\prime} + 1} \right)\frac{^{\prime}}{r}}},$

from (48) we can calculate the repair bandwidth as

$\begin{matrix}\begin{matrix}{b = {{{ab}^{\prime}\left( {q^{a} - 1} \right)} + {\left( {q^{a} - 2} \right)}}} \\{{< {\frac{}{r}\left\lbrack {n + 1 + {\left( {r - 1} \right)\left( {q^{n} - 2} \right)}} \right\rbrack}},}\end{matrix} & (54)\end{matrix}$

where the second term is the extra bandwidth compared to the originalcode.

Example 3 We take an RS(4, 2) code in GF(2¹⁶) as the original code andextend it with a=3,|E*|=7 to an RS(28, 26) code in GF(2⁴⁸) withnormalized repair bandwidth of

$\frac{b}{\left( {n - 1} \right)} < {0.65.}$

The RS(28, 26) code achieves the normalized repair bandwidth of

$\frac{b}{\left( {n - 1} \right)} < 0.54$

while it requires

=2.7×10⁸ Our scheme has a much smaller l compared while the repairbandwidth is a bit larger.

In the above theorem, extending a priori scheme to a linearly largersub-packetization and an exponentially larger code length, which meansthat for the same code length, we can have a much smallersub-packetization level.

Next, we show our second realization of the scheme in multiple cosets.Different from the previous constructions, this one allows any number ofhelpers, k≤d≤n

1. The sub-packetization size in the original code of a prior schemesatisfies

≈(n′)^(m′) when n′ grows to infinity, thus in our new code it satisfies

$ \approx {a\left( n^{\prime} \right)}^{{\overset{\Cup}{n}}^{\prime}}$

for some integer a.

Theorem 5. Let q be a prime number. There exists an RS(n, k) code overF=GF(q^(l)) of which

${l = {{asq}_{1}q_{2}\mspace{14mu} \ldots \mspace{14mu} q\frac{n}{q^{a - 1}}}},$

where q_(i) is the i-th prime number that satisfies s(q_(i)

1), s=d

k+1 and a is some integer. d is the number of helpers, k≤d≤(n

1). The average repair bandwidth is

$b = {\frac{d\; }{\left( {n - 1} \right)\left( {d - k + 1} \right)}\left\lbrack {n - 1 + {\left( {d - k} \right)\left( {q^{a} - 2} \right)}} \right\rbrack}$

measured in symbols over B=GF(q).

Proof: We first prove the case when a and l^(l) are relatively primeusing Lemma 3, the case when a and l^(l) are not necessarily relativelyprime. We use the code in as the original code, where the number ofhelpers is d0. We set n

k=n^(l)

k^(l) and calculate the repair bandwidth for d helpers from the originalcode when d^(l)=d

k+k^(l). Let us define F_(q)(α) to be the field obtained by adjoining ato the base field B. Similarly, we define F_(q)(α1, α2, . . . , αn) foradjoining multiple elements. Let be an element of order q_(i) over B.The code is defined in the field

=GF(

)=GF(q^(sq) ¹ ^(q) ² ^(. . . q) ^(n′) ⁾ which is the degree-s extensionof Fq(α1, α2, . . . αn′). The evaluation points are A^(l)={α1, α2, . . ., αn′}.

Assuming the failed node is f(αi) and the helpers are chosen from theset R′, |R′|=d′, the base for repair is

′_(i), defined as

′_(i)

_(q)(α_(j), j∈[n′], j≠i). The repair polynomials are{η_(t),p_(j)′(α_(i)), t∈[q_(i)], j∈[s]}, where

$\begin{matrix}{{{p_{j}^{\prime}(x)} = {x^{j - 1}{g^{\prime}(x)}}},{j \in \lbrack s\rbrack},{x \in ^{\prime}},} & (55) \\{{{g^{\prime}(x)} = {\prod\limits_{\alpha \in {A/{({R^{\prime}\bigcup{\{\alpha_{i}\}}})}}}\left( {x - \alpha} \right)}},{x \in {^{\prime}.}}} & (56)\end{matrix}$

and ηt∈F^(l), t∈[q_(i)], are constructed in such that {ηtp^(l)j(αi),t∈[qi], j∈[s]} forms the basis for F^(l) over F^(l)i. The repair is doneusing

$\begin{matrix}{{{tr}_{F^{\prime}/F_{i}^{\prime}}\left( {\upsilon_{\alpha_{i}}\eta_{t}{p_{j}^{\prime}\left( \alpha_{i} \right)}{f^{\prime}\left( \alpha_{i} \right)}} \right)} = {- {\sum\limits_{{\epsilon = 1},{\epsilon \neq i}}^{n^{\prime}}\; {{{tr}_{F^{\prime}/F_{i}^{\prime}}\left( {\upsilon_{\epsilon}\eta_{t}{p_{j}^{\prime}\left( \alpha_{\epsilon} \right)}{f^{\prime}\left( \alpha_{\epsilon} \right)}} \right)}.}}}} & (57)\end{matrix}$

For x∈/R^(l)∪{α_(i)}, p^(l)j(x)=0, so no information is transmitted. Theoriginal code reaches the MSR repair bandwidth

$\begin{matrix}\begin{matrix}{b^{\prime} = {\sum\limits_{e \in R^{\prime}}{{rank}_{^{\prime}i}\left( \left\{ {{{\eta_{t}{p_{j}^{\prime}\left( \alpha_{e} \right)}\text{:}t} \in \left\lbrack q_{i} \right\rbrack},{j \in \lbrack s\rbrack}} \right\} \right)}}} \\{= {\frac{d^{\prime}^{\prime}}{d^{\prime} - k^{\prime} + 1}.}}\end{matrix} & (58)\end{matrix}$

According to one or more embodiments, codes can define

=GF(q^(a))=

_(w)(α_(n+1)) where a and l^(l) are relatively prime, and α_(n+1) is anelement of order a over B. Adjoining the primitive element of F^(l) toE, then

=GF(

),

=

. The new code is defined in F. Extending the evaluation points to beA={α₁

*,α₂

*, . . . , α_(n)

*}. Since {α₁,α₂, . . . α_(n′)} are linearly independent over B, theLemma 3 can be applied and cosets are distinct. So, n=|A|=(q^(a)−1)n′.Assuming the failed node is f(α*) and α*∈α_(i)

* and the helpers are chosen from the set R, |R|=d, the base filed forrepairs is

_(i) which is defined by

_(i)

(α_(j), j∈[n+1],j≠i), for i∈[n]. The repair polynomials are defined as

{n_(t)p_(j)(x), t ∈ [q_(i)], j ∈ [s]}, where $\begin{matrix}{{{p_{j}(x)} = {x^{j - 1}{g(x)}}},{j \in \lbrack s\rbrack},{x \in },} & (59) \\{{{g(x)} = {\prod\limits_{\alpha \in {A\text{/}{({R\bigcup{\{\alpha^{*}\}}})}}}\left( {x - \alpha} \right)}},{x \in },} & (60)\end{matrix}$

and ηt is the same as that in the original code. Then, we repair thefailed node can be repaired by

$\begin{matrix}{{{{tr}_{\text{/}_{i}}\left( {\upsilon_{\alpha}*\eta_{t}{p_{j}\left( \alpha^{*} \right)}{f\left( \alpha^{*} \right)}} \right)} = {{- {\sum\limits_{{\alpha \in A},{\alpha \neq \alpha^{*}}}{{{{tr}_{\text{/}_{i}}\left( {\upsilon_{\alpha}\eta_{t}{p_{j}(\alpha)}{f(\alpha)}} \right)}.\mspace{20mu} {For}}\mspace{14mu} x}}} \in {\alpha \; ^{*}}}},{\alpha \in A^{\prime}},{{we}\mspace{14mu} {have}}} & (61) \\{\mspace{79mu} {{{p_{j}(x)} = {\gamma^{j - 1}\alpha^{j - 1}{g(x)}}},{j \in \lbrack s\rbrack},}} & (62)\end{matrix}$

for some γ∈E*. If x∈/R ↑{α*}, since g(x)=0, no information istransmitted from node x. Next, we consider all other nodes.

For x=αγ,α∈A′, since g(x) is a constant independent of j, γ∈

⊆

_(i) and η_(t), α_(i)∈

′, from Lemma 3 we have

$\begin{matrix}{{{{rank}_{_{i}}\left( \left\{ {{\eta_{t}{p_{1}(x)}},{\eta_{t}{p_{2}(x)}},\ldots \mspace{14mu},{{\eta_{t}{p_{s}(x)}\text{:}t} \in \left\lbrack q_{i} \right\rbrack}} \right\} \right)} = {{{rank}_{_{i}}\left( \left\{ {\eta_{t},{\eta_{t}\gamma \; \alpha},\ldots \mspace{14mu},{{\eta_{t}\gamma^{s - 1}\alpha^{s - 1}\text{:}t} \in \left\lbrack q_{i} \right\rbrack}} \right\} \right)} = {{{rank}_{_{i}}\left( \left\{ {\eta_{t},{\eta_{t}\alpha},\ldots \mspace{14mu},{{\eta_{t}\alpha^{s - 1}\text{:}t} \in \left\lbrack q_{i} \right\rbrack}} \right\} \right)} = {{{rank}_{_{i}^{\prime}}\left( \left\{ {\eta_{t},{\eta_{t}\alpha},\ldots \mspace{14mu},{{\eta_{t}\alpha^{s - 1}\text{:}t} \in \left\lbrack q_{i} \right\rbrack}} \right\} \right)} = {{rank}_{_{i}^{\prime}}\left( \left\{ {{\eta_{t}{p_{1}^{\prime}(\alpha)}},{\eta_{t}{p_{2}^{\prime}(\alpha)}},\ldots \mspace{14mu},{{\eta_{t}{p_{s}^{\prime}(\alpha)}\text{:}t} \in \left\lbrack q_{i} \right\rbrack}} \right\} \right)}}}}},} & (63)\end{matrix}$

which satisfies (43).

When k≤d<n

l, assuming the helpers are randomly chosen from all the remainingnodes, the average repair bandwidth for different choices of the helperscan be calculated as

$\begin{matrix}{b = {d\left\lbrack {{\frac{b^{\prime}a}{d^{\prime}} \cdot \frac{n - 1 - \left( {q^{a} - 2} \right)}{n - 1}} + {^{\prime}{a \cdot \frac{q^{a} - 2}{n - 1}}}} \right\rbrack}} & (64) \\{= {\frac{\; }{d - k + 1} + {\frac{d}{n - 1}\frac{}{d - k + 1}\left( {d - k} \right){\left( {q^{a} - 2} \right).}}}} & (65)\end{matrix}$

Here in (64) the second term corresponds to the helpers in the failednode coset, the first term corresponds to the helpers in the othercosets, and in (65) we used d′−k′=d−k.

In the case of d=n

l the repair bandwidth of the code in Theorem 5 can be directlycalculated from (48) as

$\begin{matrix}\begin{matrix}{b = {{{ab}^{\prime}\left( {q^{\alpha} - 1} \right)} + {\left( {q^{\alpha} - 2} \right)}}} \\{\left. {= {{\frac{}{r}\left( {n - 1} \right)} + {\frac{}{r}\left( {r - 1} \right)\left( {q^{\alpha} - 2} \right)}}} \right\rbrack.}\end{matrix} & (66)\end{matrix}$

In (65) and (66), the second term is the extra repair band-widthcompared to the original code.

In Theorems 4 and 5, we constructed our schemes by extending previousschemes. However, it should be noted that since we only used theproperties of the polynomials p^(l)j(x), we have no restrictions on thedimensions k^(l) of the original codes. So, in some special cases, evenif k^(l) is negative and the original codes do not exist, our theoremsstill hold. Thus, we can provide more feasible points of (n, k) usingour schemes. This is illustrated in the example below.

Example 4. Let us take the RS(12,8) code as an example. We set q=2, s=4q₁=5, q₂=9, q₃=13 and a=7. then

=2340 and

=16380. Assuming the failed node is f(α*) and α*∈α₁C, then we repair itin

₁ and set the polynomials in (59). We can easily check that when x∈α₁C,

({η_(t)p₁(x),η_(t)p₂(x), . . . , η_(t)q_(s)(x):t∈[5]})=20 and when x inother cosets,

({η_(t)p₁(x), η_(t)p₂(x), . . . , η_(t)p_(s)(x):t∈[5]})=5 Therefore; wetransmit 100 symbols in

₁, which can be normalized to

$\frac{b}{\left( {n - 1} \right)} = {0.4545.}$

As such as tradeoff is provided between l and b.

D. Numerical Evaluations and Discussions

Advantages of the embodiments may be shown relative to existing proposedschemes. Table I shows the repair bandwidth and the code length of eachscheme. For the comparison, FIGS. 4 and 5 show the performance of eachscheme when the sub-packetization changes, given (n, k)=(12, 10) and(12, 8), respectively. Considering only n

l helpers. Two single points

$\left( {{{\log_{2}()} = 53.5},{\frac{b}{\left( {n - 1} \right)} = 0.50}} \right)$

in RS (12, 10) codes and

$\left( {{{\log_{2}()} = 64.4},{\frac{b}{\left( {n - 1} \right)} = 0.25}} \right)$

in RS (12,8) codes are not shown in the figures, they can be achieved byboth our second realization in multiple cosets. We make the followingobservations.

FIG. 4 illustrates a graphical representation 400 showing normalizedrepair bandwidth values that correspond to various subpacketizationsizes as a result of implementing different repair schemes according toone or more embodiments described herein. Specifically, FIG. 4 alsodepicts a relationship between various subpacketization sizes andnormalized repair bandwidths upon the implementations of the followingrepair schemes: one coset scheme 401, two cosets scheme 402, andmultiple cosets schemes 403 and 404. FIG. 4 also depicts a Full-lengthcode scheme 405. FIG. 4 also shows a comparison of 3 schemes, q=2, n=12,k=10, r=2, x-axis is the log scale sub-packetization size, y-axis is thenormalized repair bandwidth. Moreover, the normalized repair bandwidthvalues resulting from implementation of different schemes to correct asingle erasures in one coset and multiple cosets, and multiple erasuresin one coset and multiple cosets are shown in Table III and Table IV.

For a fixed (n, k), we compare the normalized repair bandwidth b/[(n−1)

] in different sub-packetization sizes. At the outset, it is noted thatparameter

a

may be changed to adjust sub-packetization size. Regarding a one cosetscheme 401 and two cosets scheme 402 depicted in FIG. 4, the parameter ais determined by code length n, and will not be changed by increasing 1.In addition, the normalized repair bandwidth will also remain unchanged.Regarding the multiple cosets schemes 403 and 404, the parameter a maybe used to adjust the sub-packetization size. When q=2, a=1, the twoschemes in multiple cosets can coincides with prior methods. FromTheorems 4 and 5 we know that for the two schemes,

${ = {{{a \cdot r^{\frac{n}{q^{a} - 1}}}\mspace{14mu} {and}\mspace{14mu} } \approx {a \cdot \left( \frac{n}{q^{a} - 1} \right)^{(\frac{n}{q^{a} - 1})}}}},$

respectively, which means that increasing a will decrease thesub-packetization 1.

With respect to the Full-length code scheme 405, the normalized repairbandwidth increases exponentially for sub-packetization sizes rangingfrom 2 to 6. In addition, the normalized repair bandwidth approaches avalue of 1 for sub-packetization sizes ranging from 8 to 16.

FIG. 5 illustrates a graphical representation 500 showing normalizedrepair bandwidth values that correspond to various subpacketizationsizes as a result of implementing different repair schemes. Inparticular, FIG. 5 depicts a relationship between varioussubpacketization sizes and normalized repair bandwidths upon theimplementations of the following repair schemes: exemplary one cosetscheme 501, exemplary two cosets scheme 502, and exemplary multiplecosets schemes 503 and 504. FIG. 5 also depicts an exemplary Full-lengthcode scheme 505. These schemes are implemented according to one or moreembodiments described herein. FIG. 5 also illustrates a comparison of 3schemes, q=2, n=12, k=8, r=4, x-axis is the log scale sub-packetizationsize, y-axis is the normalized repair bandwidth. Moreover, thenormalized repair bandwidth values resulting from implementation ofdifferent schemes to correct a single erasures in one coset and multiplecosets, and multiple erasures in one coset and multiple cosets are shownin Table III and Table IV.

A prior scheme can achieve one tradeoff point in FIG. 5, which can beviewed as a special case of our scheme in multiple coset 1 (exemplarymultiple coset scheme 503). For fixed n, k, our schemes are better thanthe full-length code in prior schemes for all l, except when l=4, forwhich our scheme in one coset (exemplary one coset scheme 501) isidentical to the full-length code. While the repair bandwidth of thefull-length code grows with l, our schemes in one coset (exemplary onecoset scheme 501) and two cosets (exemplary two cosets scheme 502) havea constant normalized bandwidth, and our schemes in multiple cosets(exemplary multiple cosets schemes 503 and 504) have a decreasingnormalized bandwidth with l. For small l: the schemes in one coset andtwo cosets are better than those in multiple cosets; when n=12, k=10,4≤l≤48, the scheme in two cosets provides the lowest bandwidth; whenn=12, k=8, 4≤l≤768, one can show that the scheme in one coset has thesmallest bandwidth. For large 1 the first realization in multiple cosetshas better performance than the second realization in multiple cosets,but our second realization works for any number of helpers.

IV. Reed-Solomon Repair Schemes for Multiple Erasures

According to one or more embodiments, definitions of repair schemes formultiple erasures in a MDS code: can include linear repair schemedefinition and dual code repair definition. We prove the equivalence ofthe two definitions. Then, we present two schemes for repairing multipleerasures in Reed-Solomon codes, where the evaluation points are in onecoset and multiple cosets, respectively. The manner in which bandwidthis affected in multiple era

A. Definitions of the Multiple-Erasure Repair

Let us assume a scalar MDS code

over

=GF(

) has dimension k and code length n. Let a codeword be (C1, C2, . . .Cn). Without loss of generality, we assume {C1, C2, . . . , Ce} arefailed, e≤n

k, and we repair them in the base field B=GF(q), where q can be anypower of a prime number. We also assume that we use all the remainingd=n

e nodes as helpers. The following definitions are associated with singleerasure.

Definition 1. A linear exact repair scheme for multiple erasuresconsists of the following.

-   -   1) A set of queries Q_(t)⊆        for each helper C_(t), e+1≤t≤n. The helper C_(t) replies with        {γC_(t),γ∈Q_(t)}.    -   2) For each failed node C_(i),i∈[e], a linear repair scheme that        computes

$\begin{matrix}{{C_{i} = {\sum\limits_{m = 1}^{}{\lambda_{im}\mu_{im}}}},} & (67)\end{matrix}$

-   -   -   where {μ_(i1),μ_(i2), . . .            } is a basis for            over            and coefficients λ_(im)∈            are            -linear combinations of replies

$\begin{matrix}{{\lambda_{im} = {\sum\limits_{t = {e + 1}}^{n}{\sum\limits_{\gamma \in Q_{t}}{\beta_{{im}\; \gamma \; t} \cdot {{tr}_{\text{/}}\left( {\gamma \; C_{t}} \right)}}}}},} & (68)\end{matrix}$

-   -   -   with the coefficients β_(imγt)∈            . The repair bandwidth is

$\begin{matrix}{b = {\sum\limits_{t = {e + 1}}^{}{{{rank}_{}\left( Q_{t} \right)}.}}} & (69)\end{matrix}$

In the following definition, we consider el dual codewords of E, andindex them by i∈[e], j∈[

], denoted as (C_(ij1)′,C_(ij2)′, . . . , C_(ijn)′). Since they are dualcodewords, Σ_(t=1) ^(n)C_(t)C_(ijt)′=0.

Definition 2. A dual code scheme uses a set of dual codewords{(C_(ij1)′,C_(ij1)′, . . . , C_(ijn)′):i∈[e], j∈[

} that satisfies:

-   -   1) The ill rank condition: Vectors

V _(ij)=(C _(ij1) ′,C _(ij2) ′, . . . ,C _(ije)′),i∈[e],j∈[

],  (70)

-   -   -   are linearly independent over            .

    -   2) The repair bandwidth condition:

$\begin{matrix}{b = {\sum\limits_{t = {e + 1}}^{n}{{{rank}_{}\left( \left\{ {{{C_{ijt}^{\prime}\text{:}i} \in \lbrack e\rbrack},{j \in \lbrack \rbrack}} \right\} \right)}.}}} & (71)\end{matrix}$

We repair nodes [e] from the linearly independent equations

$\begin{matrix}{{{\sum\limits_{\upsilon = 1}^{e}{{tr}_{\text{/}}\left( {C_{{ij}\; \upsilon}^{\prime}C_{\upsilon}} \right)}} = {- {\sum\limits_{t = {e + 1}}^{n}{{tr}_{\text{/}}\left( {C_{ijt}^{\prime}C_{t}} \right)}}}},{i \in \lbrack e\rbrack},{j \in {\lbrack \rbrack.}}} & (72)\end{matrix}$

Here we use the same condition names as the single erasure case, but inthis section, they are defined for multiple erasures.Theorem 6. Definitions 1 and 2 are equivalent.

The equivalence of Definitions 1 and 2 follows similarly, except that weneed to first solve e failed nodes simultaneously and then find out theform of each individual failure (67).

Remark 2. In this paper, we focus on repairing RS code and apply Theorem6 to RS code. Knowing that with the polynomial p_(ij)(x)∈

[x] for which the degrees are smaller than n

k, (ν₁p_(ij)(α₁),ν₂p_(ij)(α₂), . . . ν_(n)p_(ij)(α_(n))) is the dualcodeword of RS (n, k), where ν_(i), i∈[n] are non-zero constantsdetermined by the evaluation points set A. So, in RS code, Definition 2reduces to finding polynomials pij(x) with degrees smaller than n

k. In what follows we use p_(ij)(α_(t)) to replace the dual codewordsymbol, C_(ijt)′ in Definition 2 for RS code. One can easily show thatthe constants ν_(i),i∈[n] do not affect the ranks in the full rankcondition and the repair bandwidth condition.

B. Multiple-Erasure Repair in One Coset

A scheme is provided for multiple erasures in one coset. From Theorem 6,we know that finding the repair scheme for multiple erasures in RS codeis equivalent to finding dual codewords (or polynomials) that satisfythe full rank condition and repair bandwidth condition. Given a basis{ξ1, ξ2, . . . , ξ′} for F over B, we define some matrices as below.They are used to help us check the two rank conditions according toLemmas 4 and 5. Let the evaluation points of an RS code over F be A={α1,. . . , αn}. Let p_(ij)(x), i∈[e], j∈[

] be polynomials over F, and B a subfield of F. Define

$\begin{matrix}{S_{it} = \begin{bmatrix}{{tr}_{\text{/}}\left( {\xi_{1}{p_{i\; 1}\left( \alpha_{t} \right)}} \right)} & \ldots & {{tr}_{\text{/}}\left( {\xi_{}{p_{i\; 1}\left( \alpha_{t} \right)}} \right)} \\{{tr}_{\text{/}}\left( {\xi_{1}{p_{i\; 2}\left( \alpha_{t} \right)}} \right)} & \ldots & {{tr}_{\text{/}}\left( {\xi_{}{p_{i\; 2}\left( \alpha_{t} \right)}} \right)} \\\vdots & \ddots & \vdots \\{{tr}_{\text{/}}\left( {\xi_{1}{p_{i\; }\left( \alpha_{t} \right)}} \right)} & \ldots & {{tr}_{\text{/}}\left( {\xi_{}{p_{i\; }\left( \alpha_{t} \right)}} \right)}\end{bmatrix}} & (73) \\{S\overset{\Delta}{=}\begin{bmatrix}S_{11} & S_{12} & \ldots & S_{1\; e} \\S_{21} & S_{22} & \ldots & S_{2\; e} \\\vdots & \vdots & \ddots & \vdots \\S_{e\; 1} & S_{e\; 2} & \ldots & S_{ee}\end{bmatrix}} & (74)\end{matrix}$

Lemma 4. The following two statements am equivalent:

-   -   1) vectors V_(ij)=(p_(ij)(α₁),p_(ij)(α₂), . . . ,        p_(ij)(α_(e))), i∈[e], j∈[        ] an linearly independent over        .    -   2) Matrix S in (74) has full rank.        Lemma 5. For t∈[n], consider S_(it) in (73),

$\begin{matrix}{{{rank}\left( \begin{bmatrix}S_{1t} \\S_{2t} \\\vdots \\S_{et}\end{bmatrix} \right)} = {{{rank}_{}\left( \left\{ {{{{p_{ij}\left( \alpha_{t} \right)}\text{:}i} \in \lbrack e\rbrack},{j \in \lbrack \rbrack}} \right\} \right)}.}} & (75)\end{matrix}$

-   -   Theorem 7. Let q be a prime number. There exists an RS(n,k) code        over        =GF(        ) of which n<q^(a), q^(s)≤r and a        such that the repair bandwidth for e erasures is

$b \leq {\frac{e\; }{a}\left( {n - e} \right)\left( {a - s} \right)}$

measured in symbols over

for e satisfying

$a \geq {\frac{e\left( {e - 1} \right)}{2}{\left( {a - s} \right)^{2}.}}$

-   -   Proof: We define a code over the field        =GF(        ) extended by        =GF(q^(a)), where β is the primitive element of        . The evaluation points are chosen to be A={α₁,α₂, . . . ,        α_(n)}⊆        *, which is one of the cosets in Lemma 1. Without loss of        generality we assume the e failed nodes are {α₁,α₂, . . . ,        α_(n)}. The base field is        =GF(q). Construction III: We first consider the special case        when s=a−1. In this case, inspired by [19, Proposition 1], we        choose the polynomials

$\begin{matrix}{{{p_{ij}(x)} = \frac{\delta_{i}{{tr}_{/}\left( {\frac{\mu_{j}}{\delta_{i}}\left( {x - \alpha_{i}} \right)} \right)}}{x - a_{i}}},{i \in \lbrack e\rbrack},{j \in \lbrack a\rbrack},} & (76)\end{matrix}$

-   -   where {μ₁,μ₂, . . . , μ_(a)} is the basis for        over        , and δ_(i)∈        , i∈[e], are coefficients to be determined. From [19, Theorem        3], we know that for

${a > \frac{e\left( {e - 1} \right)}{2}},$

there exists δ_(i), i∈[e] such that p_(ij)(x) satisfy the full rankcondition: the vectors V_(ij)=(p_(ij)(α₁),p_(ij)(α₂), . . . ,p_(ij)(α_(e))), i∈[e], j∈[a] are linearly independent over

and the repair bandwidth condition:

$\begin{matrix}{{\sum\limits_{t = {e + 1}}^{n}{{rank}_{}\left( \left\{ {{{{p_{ij}\left( \alpha_{t} \right)}\text{:}i} \in \lbrack e\rbrack},{j \in \lbrack a\rbrack}} \right\} \right)}} = {{\left( {n - e} \right)e} - {\frac{{e\left( {e - 1} \right)}\left( {q - 1} \right)}{2}.}}} & (77)\end{matrix}$

-   -   Then, let {η₁,η₂, . . .        } be a set of basis for        over        we have the        polynomials as {η_(w)p_(ij)(x):w∈[        /a], i∈[e], j∈[a]}. Since {η₁,η₂, . . . ,        } are linearly independent over        and for any b_(ijw)∈        , b_(ijw)p_(ij)(x)∈        , we have

$\begin{matrix}{{{\sum\limits_{i,j,w}{b_{ijw}\eta_{w}V_{ij}}} = {{0{\sum\limits_{i,j}{b_{ijw}V_{ij}}}} = 0}},{\forall{w \in {\left\lbrack \frac{}{a} \right\rbrack.}}}} & (78)\end{matrix}$

-   -   Also, we know that there does not exist nonzero b_(ijw)∈        that satisfies Σ_(i,j)b_(ijw)V_(ij)=0, so we have that vectors        {η_(w)V_(ij), w∈[        /a], i∈[e], j∈[a]} are also linearly

Construction IV: For s≤a−1, consider the polynomials

$\begin{matrix}{{{p_{ij}(x)} = {\delta_{i}^{q^{s} - 1}\mu_{j}{\prod\limits_{ɛ = 1}^{q^{s} - 1}\left( {x - \left( {\alpha_{i} - {w_{ɛ}^{- 1}\frac{\mu_{j}}{\delta_{i}}}} \right)} \right)}}},{j \in \lbrack a\rbrack},} & (80)\end{matrix}$

-   -   where {μ₁,μ₂, . . . , μ_(a)} is the basis for        over        , W={w₀=w₁,w₂, . . . , w_(q) _(s) ⁻¹} is an s-dimensional        subspace in        , s<a, q^(s)≤r, and δ_(i)∈        , i∈[e], are coefficients to be determined        -   When x=a, we have

$\begin{matrix}{{p_{ij}\left( \alpha_{i} \right)} = {\mu_{i}^{q^{s}}{\prod\limits_{ɛ = 1}^{q^{s} - 1}{w_{ɛ}^{- 1}.}}}} & (81)\end{matrix}$

Since

$\sum\limits_{ɛ = 1}^{q^{s} - 1}w_{ɛ}^{- 1}$

is a constant, from Lemma 2 we have

({p _(i1)(α₁),p _(i2)(α_(i)), . . . ,p _(ia)(α_(i))})=a.  (82)

-   -   For x≠α_(i), set x′=α_(i)−x we have

$\begin{matrix}\begin{matrix}{{p_{ij}(x)} = {\delta_{i}^{q^{s} - 1}\mu_{j}{\prod\limits_{ɛ = 1}^{q^{s} - 1}\left( {{w_{ɛ}^{- 1}\frac{\mu_{j}}{\delta_{i}}} - x^{\prime}} \right)}}} \\{= {\delta_{i}^{q^{s} - 1}\mu_{j}{\prod\limits_{ɛ = 1}^{q^{s} - 1}{\left( {w_{ɛ}^{- 1}x^{\prime}} \right){\prod\limits_{ɛ = 1}^{q^{s} - 1}\left( {\frac{\mu_{j}}{\delta_{i}x^{\prime}} - w_{ɛ}} \right)}}}}} \\{= {\left( {\delta_{i}x^{\prime}} \right)^{q^{s}}{\prod\limits_{ɛ = 1}^{q^{s} - 1}{\left( w_{ɛ}^{- 1} \right){\prod\limits_{ɛ = 0}^{q^{s} - 1}{\left( {\frac{\mu_{j}}{\delta_{i}x^{\prime}} - w_{ɛ}} \right).}}}}}}\end{matrix} & (83)\end{matrix}$

-   -   independent over        . So, from Definition 2, we know that we can recover the failed        nodes and the repair bandwidth is

$\begin{matrix}\begin{matrix}{b = {{rank}_{}\left( \left\{ {{\eta_{1}{p_{ij}(x)}},\ldots \mspace{14mu},{{\eta_{\text{/}a}{p_{ij}(x)}\text{:}i} \in \lbrack e\rbrack},{j \in \lbrack a\rbrack}} \right\} \right)}} \\{= {\frac{}{a}{{rank}_{}\left( \left\{ {{p_{ij}(x)},{i \in \lbrack e\rbrack},{j \in \lbrack a\rbrack}} \right\} \right)}}} \\{= {{\frac{}{a}\left\lbrack {{\left( {n - e} \right)e} - \frac{{e\left( {e - 1} \right)}\left( {q - 1} \right)}{2}} \right\rbrack}.}}\end{matrix} & (79)\end{matrix}$

As,

${g(y)} = {\prod\limits_{f = \bigcap}^{q^{s} - 1}\left( {y - w_{ɛ}} \right)}$

is a linear mapping from E to itself with dimension a-s over B. Since

$\left( {\delta_{i}x^{\prime}} \right)^{q^{s}}{\prod\limits_{ɛ = 1}^{q^{s} - 1}\left( w_{ɛ}^{- 1} \right)}$

is a constant independent of J, we have

({p _(i1)(x),p _(i2)(x), . . . ,p _(ia)(x)})≤a−s,  (84)

-   -   which means that p_(ij)(x) can be written as

$\begin{matrix}{{{p_{ij}(x)} = {\delta_{i}^{q^{s}}{\sum\limits_{\upsilon = 1}^{\alpha - s}{\rho_{j\; \upsilon}\lambda_{\upsilon}}}}},} & (85)\end{matrix}$

where {λ₁,λ₂, . . . λ_(a−s)} are linearly independent over B, ρ_(jν)∈

and thy are determined by δ_(i), μ_(j) and x−α_(i).

From Lemma 4, we know that if the matrix S in (74) has full rank, thenwe can recover the e erasures. It is difficult to directly discuss therank of the matrix, but assume that the polynomials above satisfy thefollowing two conditions:

-   -   1) S_(ii), i∈[e] are identity matrices.    -   2) For any fixed i∈[e],

S _(it) ·S _(iy) =

,i>y>t.  (86)

Then, it is easy to see that through Gaussian elimination, e cantransform the matrix S^(T) to an upper triangle block matrix, which hasidentify matrices in the diagonal. Hence, S has full rank. Here,selecting {ξ₁,ξ₂, . . . ,

} to be the dual basis of

$\begin{matrix}{{\left\{ {{\mu_{1}^{q^{s}}{\underset{ɛ = 1}{\prod\limits^{q^{s} - 1}}w_{ɛ}^{- 1}}},{\mu_{2}^{q^{s}}{\prod\limits_{ɛ = 1}^{q^{s} - 1}w_{ɛ}^{- 1}}},\ldots \mspace{14mu},{\mu_{}^{q^{s}}{\prod\limits_{ɛ = 1}^{q^{s} - 1}w_{ɛ}^{- 1}}}} \right\},{so}}{{{tr}_{\text{/}}\left( {\xi_{m}{p_{ij}\left( \alpha_{i} \right)}} \right)} = \left\{ \begin{matrix}{0,{m \neq j},} \\{1,{m = {j.}}}\end{matrix} \right.}} & (87)\end{matrix}$

Therefore, Sii, i∈[e] are identity matrices. We set δ1=1, andrecursively choose δi after choosing {δ1, δ2, . . . , δi

1} to satisfy (86). Define δ_(i)′=δ_(i) ^(q)*, and c_(mp) to be the (m,p)-th element in Sty for m,p∈[a]. Formula (86) can be written as

$\begin{matrix}{{{\sum\limits_{m = 1}^{a}{c_{mp}{{tr}_{\text{/}}\left( {\xi_{m}{p_{ij}\left( \alpha_{t} \right)}} \right)}}} = {{\sum\limits_{m = 1}^{a}{c_{mp}{\sum\limits_{\upsilon = 1}^{a - s}{b_{j\; \upsilon}{{tr}_{\text{/}}\left( {\xi_{m}\delta_{i}^{\prime}\lambda_{\upsilon}} \right)}}}}} = 0}},{\forall{j \in \lbrack a\rbrack}},} & (88)\end{matrix}$

-   -   where λ_(ν), ν∈[a−s] are determined by

$\begin{matrix}{{{\sum\limits_{m = 1}^{a}{c_{mp}{{tr}_{\text{/}}\left( {\xi_{m}{p_{ij}\left( \alpha_{t} \right)}} \right)}}} = {{\sum\limits_{m = 1}^{a}{c_{mp}{\sum\limits_{\upsilon = 1}^{a - s}{b_{j\; \upsilon}{{tr}_{\text{/}}\left( {\xi_{m}\delta_{i}^{\prime}\lambda_{\upsilon}} \right)}}}}} = 0}},{\forall{j \in \lbrack a\rbrack}},.} & (88)\end{matrix}$

Equation (88) is satisfied if

$\begin{matrix}{{{\sum\limits_{m = 1}^{a}{c_{mp}{{tr}_{\text{/}}\left( {\xi_{m}\delta_{i}^{\prime}\lambda_{\upsilon}} \right)}}} = 0},{\upsilon \in \left\lbrack {a - s} \right\rbrack},{p \in {\lbrack a\rbrack.}}} & (89)\end{matrix}$

As a special case of Lemma 5, we have

rank(S _(ty))=

({p _(tj)(α_(y)),j∈[

]}).  (901)

Then, from (84) we know that the rank of Sty is at most a

s, which means in (89) we only need to consider p corresponding to theindependent a

s columns of Sty. So, (89) is equivalent to (a

s)² linear requirements. For δ_(i)′∈

, we can view it as a unknowns over B, and we have

$\begin{matrix}{{\frac{\left( {{2\; e} - i} \right)\left( {i - 1} \right)}{2}\left( {a - s} \right)^{2}} \leq {\frac{e\left( {e - 1} \right)}{2}\left( {a - s} \right)^{2}}} & (91)\end{matrix}$

linear requirements over B according to (86). Also knowing δ_(i)′, wecan solve δ_(i)=δ_(i) ^(q) ^(l) =δ′_(i) ^(q) ^(l−s) . Therefore, we canfind appropriate {δ1, δ2, . . . δe} to make matrix S full rank when

$\begin{matrix}{a \geq {\frac{e\left( {e - 1} \right)}{2}{\left( {a - s} \right)^{2}.}}} & (92)\end{matrix}$

Then, let {η1, η2, . . . , η′/a} be a basis for F over E, we have the elpolynomials as {η_(w)p_(ij)(x), w∈[

/a], i∈[e], j∈[a]}. Similar to Construction III, we know that vectors{η_(w)V_(ij), w∈[

/a], i∈[e], j∈[a]} are linearly independent over B. Therefore, we canrecover the failed nodes and the repair bandwidth is

$\begin{matrix}\begin{matrix}{b = {{rank}_{}\left( \left\{ {{\eta_{1}{p_{ij}(x)}},\ldots \mspace{14mu},{{\eta_{\text{/}a}{p_{ij}(x)}\text{:}i} \in \lbrack e\rbrack},{j \in \lbrack a\rbrack}} \right\} \right)}} \\{= {{\frac{}{a}{{rank}_{}\left( \left\{ {{{{p_{ij}(x)}\text{:}i} \in \lbrack e\rbrack},{j \in \lbrack a\rbrack}} \right\} \right)}} \leq {\frac{\mathcal{e}}{a}\left( {n - e} \right){\left( {a - s} \right).}}}}\end{matrix} & (93)\end{matrix}$

In our scheme, we have constructions for arbitrary a, s, such that a|

, s≤a−1, while the existing schemes mainly considered the special casel=a. It should be noted that the scheme in [19] can also be used in thecase of s=a

l over E with repair bandwidth

${\left( {n - e} \right)e} - {\frac{{e\left( {e - 1} \right)}\left( {q - 1} \right)}{2}.}$

And, with lla copies of the code, it can also reach the same repairbandwidth of our scheme. However, by doing so, the code is a vectorcode, but our scheme constructs a scalar code.

C. Multiple-Erasure Repair in Multiple Cosets

Recall that the scheme in Theorem 5 for a single erasure is a smallsub-packetization code with small repair bandwidth for any number ofhelpers. When there are e erasures and d helpers, e≤n

k, k≤d≤n

e, we can recover the erasures one by one using the d helpers. However,the repaired nodes can be viewed as additional helpers and thus we canreduce the total repair bandwidth. Finally, for every helper, thetransmitted information for different failed nodes has some overlap,resulting in a further bandwidth reduction.

FIG. 6 shows erasure locations in various data packets according to oneor more embodiments. In particular, FIG. 6 depicts the number oferasures or failures present in various cosets. In one embodiment, anerasure may be present in each of h₁ cosets 601, h₂ cosets 602, andh_(e) cosets 603.

According to one embodiment, using an original code, the code isextended to a new code with evaluation points as in (44). If a helper isin the same coset as any failed node, it transmits naively its entiredata; otherwise, it transmits the same amount as the scheme in theoriginal code. After the extension, the new construction decreases thesub-packetization size for fixed n, and the bandwidth is only slightlylarger than the original code. The location of the e erasures aredescribed by h_(i), i∈[e], where 0≤h_(i)≤e, h₁≥h₂≥ . . . ≥h_(e), Σ_(i=1)^(e)h_(i)=e. We assume the erasures are located in h₁ cosets 601, andafter removing one erasure in each coset, the remaining erasures arelocated in h₂ cosets 602. Then, for the remaining erasures, removing onein each coset, we get the rest of erasures in h_(e) cosets, and so on.FIG. 6 also shows the erasure locations described above.

In our scheme, we first repair h₁ failures, one from each of the h₁cosets. Then, for 2≤i≤e, we repeat the following: After repairing h₁,h₂, . . . ,

₁ failures, we view these repaired nodes as helpers and repair next hifailures, one from each of the h cosets. The repair bandwidth of thescheme is showed in the following theorem.

-   -   Theorem 8. Let q be a prime number. There exists an RS(n,k) code        over        =GF(        ) for which

${ = {{asq}_{1}q_{2}\mspace{14mu} \ldots \mspace{14mu} q\frac{n}{q^{a} - 1}}},$

where q_(i) is the i-th prime number that satisfies s|(q_(i)−1),s=(n−k)! and a is an integer. For e erasures and d helpers, e≤n−k,k≤d≤n−e the average repair bandwidth measured in symbols over

is

$\begin{matrix}{{b \leq {\frac{\; }{\left( {n - e} \right)}\left\lbrack {\left( {{h_{1}\left( {q^{a} - 1} \right)} - e} \right) + {\left( {n - {h_{1}\left( {q^{a} - 1} \right)}} \right){\sum\limits_{i = 1}^{e}\frac{h_{i}}{d - k + {\sum_{\upsilon = 1}^{i}h_{\upsilon}}}}}} \right\rbrack}},} & (94)\end{matrix}$

Where h_(i), i∈[e] are the parameters that define the location oferasures in FIG. 6

-   -   Proof: We first prove the case when a and        are relatively prime using Lemma 3, the case when a and        are not necessarily relatively prime are proved in Appendix A.        We use the code in [22] as the original code. Let        _(q)(α) be the field obtained by adjoining α to the base field        =GF(q). Similarly let        _(q)(α₁,α₂, . . . , α_(n)) be the field for adjoining multiple        elements. Let α_(i) be an element of order q_(i) over        and h be the number of erasures in the original code. The        origial code is defined in the field        ′=GF(        )=GF(q^(sq) ¹ ^(q) ² ^(. . . q) ^(n′) ⁾, which is the degree-s        of extension of        (α₁,α₂, . . . , α_(n′)). The evaluation points are A′={α₁,α₂, .        . . , α_(n)}. The subfield        ′_([h]) is defined as        ′_([h])=        _(q)(α_(j), j=h+1, h+2, . . . , n′), and        ′_(i) is defined as        q(αj≠i, j∈[n′]).    -   In the original code, we assume without loss of generality that        there are h failed nodes f′(α₁), f′(α₂), . . . , f′(α_(h)).        Consider the polynomials for failed node f′(α_(i)), 1≤i≤h. as

p _(ij)′(x)=x ^(j−1) g _(i)′(x),j∈[s _(i)],x∈

′,  (95)

where

$\begin{matrix}{{{g_{i}^{\prime}(x)} = {\prod\limits_{\alpha \in {A^{\prime}/{({R^{\prime}\bigcup{\{{\alpha_{i},\alpha_{i + 1},{\ldots \; \alpha_{h}}}\}}})}}}^{\;}\; \left( {x - \alpha} \right)}},{x \in ^{\prime}},} & (96)\end{matrix}$

-   -   for R′⊆A′, |R|=d′ being the set of helpers. The set of reapair        polynomials are {η_(it)p_(ij)′(z), i∈[h], j∈[s_(i)], t∈

$\left. \left\lbrack \frac{{sq}_{i}}{s_{i}} \right\rbrack \right\},$

where η_(it)∈

′ are constructed in [22] to ensure that {η_(it)p_(i1)′(α_(i)),η_(it)p_(i2)′(α_(i)), . . . , η_(it)p_(is) _(i) ′(α_(i))} forms thebasis for

′ over

′_(i).

-   -   -   Then, the failed nodes are repaired one by one from

$\begin{matrix}{{{tr}_{^{\prime}/_{i}^{\prime}}\left( {\upsilon_{\alpha},{\eta_{it}{p_{ij}^{\prime}\left( \alpha_{i} \right)}{f^{\prime}\left( \alpha_{i} \right)}}} \right)} = {- {\sum\limits_{{e = 1},{e \neq i}}^{n}{{{tr}_{^{\prime}/_{i}^{\prime}}\left( {\upsilon_{e},{\eta_{it}{p_{ij}^{\prime}\left( \alpha_{e} \right)}{f^{\prime}\left( \alpha_{e} \right)}}} \right)}.}}}} & (97)\end{matrix}$

-   -   For x∉R∪{α₁, α_(i+1), . . . , α_(h)}, p_(ij)′(z)=0 and no        information is transmitted. Once f′(a) is recovered, it is        viewed as a new helper for the failures i+1,i+2, . . . , h.        -   Since            _([h])≤            ′_(i), the information transmitted from the helper α_(t) can            be represented as

$\begin{matrix}{{{{tr}_{^{\prime}/_{i}^{\prime}}\left( {\upsilon_{e},{\eta_{it}{p_{ij}^{\prime}\left( \alpha_{e} \right)}{f^{\prime}\left( \alpha_{e} \right)}}} \right)} = {{{tr}_{^{\prime}/_{i}^{\prime}}\left( {\xi_{im}^{\prime}{\sum\limits_{m = 1}^{q_{i}^{\prime}}{{tr}_{_{i}^{\prime}/_{\lbrack h\rbrack}^{\prime}}\left( {\upsilon_{\epsilon}\eta_{it}\xi_{im}{p_{ij}^{\prime}\left( \alpha_{\epsilon} \right)}{f^{\prime}\left( \alpha_{\epsilon} \right)}} \right)}}} \right)} = {\sum\limits_{m = 1}^{q_{i}^{\prime}}{\xi_{im}^{\prime}{{tr}_{_{i}^{\prime}/_{\lbrack h\rbrack}^{\prime}}\left( {\upsilon_{\epsilon}\eta_{it}\xi_{im}{p_{ij}^{\prime}\left( \alpha_{\epsilon} \right)}{f^{\prime}\left( \alpha_{\epsilon} \right)}} \right)}}}}},} & (98)\end{matrix}$

-   -   Where

${q_{i}^{\prime} = \frac{q_{1}q_{2}\ldots \; q_{h}}{q_{i}}},$

{ξ_(i1),ξ_(i2), . . . ξ_(iq) _(i) _(′)} and {ξ_(i1)′,ξ_(i2)′, . . .ξ_(iq) _(i) _(′)} are the dual basis for

′ over

_([h]). We used the fact that

, (

(⋅))=

(⋅), for

′_([h])≤

F′_(i)≤

′.

The original code satisfies the full rank condition for every i E [h],and each helper a transmits

$\begin{matrix}{{{rank}_{_{\lbrack h\rbrack}^{\prime}}\left( \left\{ {{{\eta_{ih}\xi_{im}{p_{ij}^{\prime}\left( \alpha_{\epsilon} \right)}\text{:}\mspace{11mu} i} \in {\lbrack h\rbrack j} \in \left\lbrack s_{i} \right\rbrack},{t \in \left\lbrack \frac{sq}{s_{i}} \right\rbrack},{m \in \left\lbrack q_{i}^{\prime} \right\rbrack}} \right\} \right)} = {{{rank}_{_{\lbrack h\rbrack}^{\prime}}\left( \left\{ {{{\eta_{it}\xi_{im}\text{:}\mspace{11mu} i} \in \lbrack h\rbrack},{t \in \left\lbrack \frac{{sq}_{i}}{s_{i}} \right\rbrack},{m \in \left\lbrack q_{i}^{\prime} \right\rbrack}} \right\} \right)} = \frac{h\; ^{\prime}}{\left( {d^{\prime} - k^{\prime} + h} \right){\prod_{\upsilon = {h + 1}}^{n^{\prime}}\; p_{\upsilon}}}}} & (99)\end{matrix}$

symbols over

′_([h]), which achieves the MSR bound.

In our new code, we extend field to

=GF (

),

=

by adjoining an order-a element α_(n+1) to F. We set d−k=d′−k′. The newevaluation points consist of A={α₁

*,α₂

*, . . . , α_(n)′

*},

=GF(q^(a))=

_(q)(α_(n+1)). The subfield F[h] is defined by adjoining α_(n+1) to

′_([h]), and

_(i), is defined as

_(q)(α_(j), j≠i, j∈[n+1]).

Assume first that each coset contains at most one failure, and there areh failures in total. We assume without loss of generality that theevaluation points of the h failed nodes are in {α₁

*,α₂

*, . . . , α_(h)

*} and they are α₁γ₁,α₂γ₂, . . . , α_(h)γ_(h) for some γ_(w)∈

, w∈[h]. Let the set of helpers be R⊆A, |R|=α. We define the polynomialsas

p _(ij)(x)=x ^(j−1) g _(i)(x),j∈[s _(i)],x∈

,  (100)

where

$\begin{matrix}{{{g_{i}(x)} = {\prod\limits_{\alpha \in {A/{({R\bigcup{\{{{\alpha_{i}\gamma_{i}},{\alpha_{i + 1}\gamma_{i + 1}},{\ldots \; \alpha_{h}\gamma_{h}}}\}}})}}}^{\;}\; \left( {x - \alpha} \right)}},{x \in {.}}} & (101)\end{matrix}$

The set of repair polynomials are {η_(it)p_(ij)(x), i∈[h], j∈whereη_(it)∈

are the dame as the original construction. We use field F_(i) as thebase filed for the repair.

$\begin{matrix}{{{tr}_{/_{i}}\left( {\upsilon_{\alpha_{i}\gamma_{i}}\eta_{it}{p_{ij}\left( {\alpha_{i}\gamma_{i}} \right)}{f\left( {\alpha_{i}\gamma_{i}} \right)}} \right)} = {- {\sum\limits_{{a \in A},{\alpha \neq {\alpha_{i}\gamma_{i}}}}{{{tr}_{/_{i}}\left( {\upsilon_{\alpha}\eta_{it}{p_{ij}(\alpha)}{f(\alpha)}} \right)}.}}}} & (102)\end{matrix}$

If z∈R∪{α_(i)γ_(i),α_(i+1)γ_(i+1), . . . , α_(h)γ_(h)}, p_(ij)(x)=0 andno information is transmitted. Next, we consider all other nodes. Ifx=α_(i)γ for some γ∈

, we have

p _(ij)(x)=γ^(j−1)α_(i) ^(j−1) g _(i)(x).  (103)

Since η_(it), α_(i)∈

′ and g_(i)(x) is a constant independent of j, we have

$\begin{matrix}{{{rank}_{_{i}}\left( \left\{ {{\eta_{it}{p_{i\; 1}(x)}},\ldots \;,{{\eta_{it}{p_{{is}_{i}}(x)}\text{:}\mspace{11mu} t} \in \left\lbrack \frac{{sq}_{i}}{s_{i}} \right\rbrack}} \right\} \right)} = {{{rank}_{_{i}}\left( \left\{ {\eta_{it},\ldots \;,{\eta_{it}{\alpha_{i}^{s - 1}:{t \in \left\lbrack \frac{{sq}_{i}}{s_{i}} \right\rbrack}}}} \right\} \right)} = {{rank}_{_{i}^{\prime}}\left( \left\{ {{\eta_{it}{p_{i\; 1}^{\prime}\left( \alpha_{i} \right)}},\ldots \;,{{\eta_{it}{p_{{is}_{i}}^{\prime}\left( \alpha_{i} \right)}\text{:}\mspace{14mu} t} \in \left\lbrack \frac{{sq}_{i}}{s_{i}} \right\rbrack}} \right\} \right)}}} & (104)\end{matrix}$

Which indicates the full rank. Note that the last equation follows fromLemma 3. As a result we can recover the failed nodes and each helper inthe cosets containing the failed nodes transmit l symbols in B.

For x=α_(ϵ)γ,ϵ>h, since

_([h]) is a subfield of

_(i) and from Lemma 3 we know that {ξ_(i1),ξ_(i2), . . . , ξ_(iq) _(i)′} and {ξ_(i1)′,ξ_(i2)′, . . . , ξ_(iq) _(i) ′} are also the dual basisfor

_(i) over

_([h]), then similar to (98) we have

$\begin{matrix}{{{tr}_{/_{i}}\left( {\upsilon_{\alpha}\eta_{it}{p_{ij}(x)}{f(x)}} \right)} = {\sum\limits_{m = 1}^{q_{i}^{\prime}}{\xi_{im}^{\prime}{{{tr}_{/_{\lbrack h\rbrack}}\left( {\upsilon_{\epsilon}\eta_{it}\xi_{im}{p_{ij}(x)}{f(x)}} \right)}.}}}} & (105)\end{matrix}$

Using the fact that gi(x) is a constant independent of j, x∈

_([h]) and η_(it)ξ_(im)∈

′, from Lemma 3 we know that

$\begin{matrix}{{{{rank}_{_{\lbrack h\rbrack}}\left( \left\{ {{{\eta_{it}\xi_{im}{p_{ij}(x)}\text{:}\mspace{11mu} i} \in \lbrack h\rbrack},{j \in \left\lbrack s_{i} \right\rbrack},{t \in \left\lbrack \frac{{sq}_{i}}{s_{i}} \right\rbrack},{m \in \left\lbrack q_{i}^{\prime} \right\rbrack}} \right\} \right)} = {{{rank}_{_{\lbrack h\rbrack}}\left( \left\{ {{{\eta_{it}\xi_{im}\text{:}\mspace{11mu} i} \in \lbrack h\rbrack},{t \in \left\lbrack \frac{{sq}_{i}}{s_{i}} \right\rbrack},{m \in \left\lbrack q_{i}^{\prime} \right\rbrack}} \right\} \right)} = {{{rank}_{_{\lbrack h\rbrack}^{\prime}}\left( \left\{ {{{\eta_{it}\xi_{im}\text{:}\mspace{11mu} i} \in \lbrack h\rbrack},{t \in \left\lbrack \frac{{sq}_{i}}{s_{i}} \right\rbrack},{m \in \left\lbrack q_{i}^{\prime} \right\rbrack}} \right\} \right)} = {{{rank}_{_{\lbrack h\rbrack}^{\prime}}\left( \left\{ {{{\eta_{it}\xi_{im}{p_{ij}^{\prime}\left( \alpha_{\epsilon} \right)}\text{:}\mspace{11mu} i} \in \lbrack h\rbrack},{j \in \left\lbrack s_{i} \right\rbrack},{t \in \left\lbrack \frac{{sq}_{i}}{s_{i}} \right\rbrack},{m \in \left\lbrack q_{i}^{\prime} \right\rbrack}} \right\} \right)} = \frac{h\; ^{\prime}}{\left( {d - k + h} \right){\prod_{\upsilon = {h + 1}}^{n^{\prime}}\; p_{\upsilon}}}}}}},} & (106)\end{matrix}$

where the last equality follows from (99) and d′−k′=d−k. So, each helperin the other cosets transmits

$\frac{h\; }{d - k + h}$

symbols over B.

Using the above results, we calculate the repair bandwidth in two steps.

Step 1. We first repair h1 failures, one from each of the h1 cosets.From (104), we know that in the h cosets containing the failed nodes, wetransmit ′ symbols over B. By (106), for each helper in other cosets, wetransmit

$\frac{h\; }{d - k + h_{1}}$

symbols over B.

Step 2. For 2≤i≤e, repeat the following. After repairing h1, h2, . . . ,hi

1 failures, these nodes can be viewed as helpers for repairing next hifailures, one from each of the hi cosets. So, we have

$d + {\sum\limits_{\upsilon = 1}^{i - 1}h_{\upsilon}}$

helpers for the hi failures. v=1

For the helpers in the h cosets containing the failed nodes, we alreadytransmit ′ symbols over B in Step 1 and no more information needs to betransmitted. For each helper in other cosets, we transmit

$d + {\sum\limits_{\upsilon = 1}^{i - 1}h_{\upsilon}}$

symbols over B.

Thus, we can repair all the failed nodes. The repair bandwidth can becalculated as (94).

Suppose that e failures are to be recovered. Compared to the naivestrategy which always uses d helpers to repair the failures one by one,our scheme gets a smaller repair bandwidth since the recovered failuresare viewed as new helpers and we take advantage of the overlappedsymbols for repairing different failures.

In the case when n>>e(q^(a)−1), or when we arrange nodes with correlatedfailures in different cosets, we can assume that all the erasures are indifferent cosets, h1=e, h2=h3= . . . =he=0. For example, if correlatedfailures tend to appear in the same rack in a data center, we can assigneach node in the rack to a different coset. Under such conditions, wesimplify the repair bandwidth as

$\begin{matrix}{b \leq {\frac{d}{n - e}\frac{\mathcal{e}}{d - k + e}{\left( {n - e + {\left( {d - k} \right)\left( {q^{a} - 2} \right)}} \right).}}} & (107)\end{matrix}$

Indeed, one can examine the expression of (94). With the constraint thatΣ_(i=1) ^(e) h_(i)=e, the first term is an increasing function of h1 andthe second term

$\left( {n - {h_{1}\left( {q^{a} - 1} \right)}} \right){\sum_{i = 1}^{e}\frac{h_{i}}{d - k + {\sum_{\upsilon = 1}^{i}h_{\upsilon}}}}$

is a decreasing function of h1. Under the assumption that n is large,the second term dominates, and increasing h1 reduces the total repairbandwidth b. Namely, h1=e corresponds to the lowest bandwidth for largecode length.

In particular, when d=n

e, h1=e, we have

$\begin{matrix}{{b = {{\frac{\mathcal{e}}{n - k}\left( {n - e} \right)} + {\frac{\mathcal{e}}{n - k}\left( {n - k - e} \right)\left( {q^{a} - 2} \right)}}},} & (108)\end{matrix}$

where the second term is the extra repair bandwidth com-pared with theMSR bound.

Numerical Evaluations and Discussions

In this subsection, we compare our schemes for multiple erasures withprevious results, including separate repair and schemes. Repairingmultiple erasures simultaneously can save repair bandwidth compared torepairing erasures separately. Assuming e failures happen one by one,and the rest of n

1 nodes are available as helpers initially when the first failureoccurs. We can either repair each failure separately using n

1 helpers, or wait for e failures and repair all of them simultaneouslywith n

e helpers. Table III shows a comparison. For our scheme in one coset,separate repair needs a repair bandwidth of

${\frac{\mathcal{e}}{a}\left( {n - 1} \right)\left( {a - s} \right)},$

symbols in B, simultaneous a repair requires a bandwidth of

$\frac{\mathcal{e}}{a}\left( {n - e} \right){\left( {a - s} \right).}$

For our scheme in multiple cosets, we can repair the failures separatelyby n

1 helpers with the bandwidth of

${\frac{\mathcal{e}}{n - k}\left\lbrack {n - 1 + {\left( {n - k - 1} \right)\left( {q^{a} - 2} \right)}} \right\rbrack},$

and with simultaneous repair we can achieve the bandwidth of

${\frac{\mathcal{e}}{n - k}\left\lbrack {n - e + {\left( {n - k - e} \right)\left( {q^{a} - 2} \right)}} \right\rbrack}.$

One can see that in both constructions, simultaneous repair outperformsseparate repair.

Next we compare our scheme for multiple erasures with the existingschemes.

FIG. 4 shows the normalized repair bandwidth for different schemes whenn=16, k=8, e=2, q=2. Table IV shows the comparison when n=64, k=32, e=2,q=2. We make the following observations:

-   -   1) For fixed (n, k) and our scheme with multiple cosets, we use        the parameter a to adjust the sub-packetization size. From        Theorem 8, we know that l≈a

$\left( \frac{n}{q^{a} - 1} \right)^{(\frac{n}{q^{a} - 1})}$

which means that increasing a will de-crease the sub-packetization ′. Inour schemes with one coset and two cosets, the parameter a is determinedby the code length n, so increasing l will not change a or thenormalized repair bandwidth.

-   -   2) When l grows larger (4<        <2.1=10⁷ in FIG. 4 6<        <3.3×10¹¹ in Table IV) the embodiments have the smallest repair        bandwidth.    -   3) For extremely large        in FIG. 4,        ≥3.3×10¹¹ in Table V) the embodiments in multiple cosets have        the smallest repair bandwidths.

Table III illustrates Repair bandwidth of different schemes for eerasures

TABLE III repair bandwidth number of helpers Single-erasure repair inone coset (separate repair)$\frac{e}{a}\left( {n - 1} \right)\left( {a - s} \right)$ n − 1Multiple-erasure repair in one coset (simultaneous repair)$\frac{e\; }{a}\left( {n - e} \right)\left( {a - s} \right)$ n − eSingle-erasure repair in mulitple cosets (separate repair)$\frac{e\; }{n - k}\left\lbrack {n - 1 + {\left( {n - k - 1} \right)\left( {q^{a} - 2} \right)}} \right\rbrack$n − 1 Multiple-erasure repair in mulitple cosets (simultaneous$\frac{e\; }{n - k}\left\lbrack {n - e + {\left( {n - k - e} \right)\left( {q^{a} - 2} \right)}} \right\rbrack$n − e repair)

Table IV shows normalized repair bandwidth,

$\left. \frac{b}{\left( {n - e} \right)} \right)$

for different schemes when n=64, k=32, e=2, q=2.

TABLE IV

 = 6

 = 7

 = 8

 = 9 . . .

 = 3.6 × 10

 = 3.3 × 10¹¹

 = 3.9 × 10

Normalized bandwidth 0.42 0.50 0.52 0.52 . . . 0.52 0.52 0.52 for Scheme1 in [19] Normalized bandwidth 0.49 0.49 0.49 0.49 . . . 0.49 0.49 0.49for our scheme in one coset Normalized bandwidth 0.52 0.48 0.0625* forour scheme in multiple cosets

indicates data missing or illegible when filed

FIG. 7 illustrates a graph representation showing normalized repairbandwidth values that correspond to various subpacketization sizes as aresult of implementing different repair schemes that are in accordancewith one or more embodiments described herein. Specifically, FIG. 7depicts a relationship between various subpacketization sizes andnormalized repair bandwidths upon the implementations of the followingrepair schemes: current scheme in one coset 701, current scheme inmultiple cosets 702, MBW Scheme 703. The MBW scheme 703 relates toscheme 2, which is designed by Mardia, Bartan, and Wootters. FIG. 7 alsoillustrates a comparison of the schemes, q=2, n=16, k=8, e=2, x-axis isthe log scale sub-packetization size, y-axis is the normalized repairbandwidth. Moreover, the normalized repair bandwidth values resultingfrom implementation of different schemes to correct a single erasures inone coset and multiple cosets, and multiple erasures in one coset andmultiple cosets are shown in Table III and Table IV.

As described above, three Reed-Solomon code repair schemes are providedthat include a tradeoff between the sub-packetization size and therepair bandwidth. Our schemes (current scheme in one coset 701 andcurrent scheme in multiple cosets 702) choose the evaluation points ofthe Reed-Solomon code from one, two, or multiple cosets of themultiplicative group of the underlying finite field. For a singleerasure, when the sub-packetization size is large, the scheme inmultiple cosets has better performance, it approaches the MSR bound.When sub-packetization size is small, the scheme in one coset hasadvantages in repair bandwidth. The scheme in two cosets has smallerrepair bandwidth with certain parameters in between the other two cases.For multiple erasures, our scheme in one coset has constructions forarbitrary redundancy n

k and our scheme in multiple cosets reduced the sub-packetization sizeof an MSR code. The two schemes together provided a set of tradeoffpoints and we observe similar tradeoff characteristics as in the singleerasure case. In spite of several tradeoff points we provided in thispaper, the dependence of the sub-packetization size versus the repairbandwidth is still an open question.

FIG. 8 illustrates a process 800 for error correction of distributeddata based on modifying a Reed-Solomon correction code according to oneor more embodiments described herein.

According to one embodiment, process 800 may be initiated at block 801when a device (e.g., repair device 120 depicted in FIG. 1) receives adata request from one or more client devices (e.g., one or more ofclient devices 115 _(1-n)).

At block 802, process 800 includes receiving a first correction code fora plurality of data fragments stored in a plurality of storage nodes.The first correction code is a Reed-Solomon code that has a data symbolfor each of the plurality of storage nodes and is represented as apolynomial over a finite field with a sub-packetization size.

At block 803, process 800 includes constructing a second correction codethat is constructed in response to at least one unavailable storage nodeof the plurality of storage nodes. The second correction code isrepresented as a second polynomial over a second finite field and has anincreased subpacketization size relative to the first polynomial. In oneembodiment, the second correction code is a new Reed-Solomon code. Thesecond correction code may be constructed using one or more operationsdiscussed relative to block 210 in process 200 described above.Moreover, it is noted that the unavailability may be due to an erasureerror in one of the plurality of data fragments. An erasure error mayoccur if a storage location of a particular symbol included in one ofthe data fragments stored in the storage nodes is unknown.

At block 804, process 800 includes performing erasure repair using thesecond correction code. In one embodiment, the second correction code isapplied to available data fragments of the plurality of data fragmentsfor erasure repair. Additionally, the erasure error utilizes at leastone coset of a multiplicative group of the second finite field. In oneembodiment, repairing the erasure includes using a first dual codewordand a second dual codeword as repair polynomials. In this embodiment, atrace function is combined with the dual code words to generatefragments. In one embodiment, the trace function is used to obtainsubfield symbols, as described in the

Preliminaries

section below. Moreover, the use of the trace function in combinationwith dual codewords as described in the

Reed-Solomon Repair Schemes for Multiple Erasures

section. The erasure repair may be performed using one or moreoperations discussed relative to block 215 in process 200 describedabove.

At block 805, process 800 includes outputting a corrected data fragmentbased on the erasure repair that is performed in block 215. In oneembodiment, errors in one or more of the plurality of data fragments arecorrected and the data fragments are restored in their entirety suchthat all data fragments are accessible.

Repairing Reed-Solomon (14, 10) in GF(2⁸)

Another embodiment is directed to providing an algorithm to repair asingle node failure in the (14,10) Reed-Solomon code over GFL(2⁸) in asystem (e.g., Facebook

s f4 system). Embodiments are directed to an algorithm that can repairany failed node and requires only 4 bits from each remaining node

Repair Algorithm for RS (14, 10) Code

Notation: [a] represents {1, 2, . . . , a} for a positive integer a.

The RS (14, 10) code has n=14 codeword symbols and k=10 informationsymbols. In a storage system, different symbols are stored in differentnodes. The symbols are over the finite field

=GF(2⁸)={0,1,β,β², . . . β²⁵⁴}, where β is the root of 1+x²+x³+x⁴+x⁸=0.For the given information symbols u_(j)∈

, j=0,1, . . . , 19, let f be the polynomial

${f(x)} = {\sum\limits_{j = 0}^{9}{u_{j}{x^{j}.}}}$

RS codeword symbols are evaluations of the polynomial f and erasures canbe corrected using interpolation. By restricting the evaluation pointsto a subfield, one obtains a low repair bandwidth given the field size.Consider the subfield

=GF(2⁴)={0,1,β¹⁷,β^(17·2), . . . , β^(17·14)} of F. A set of evaluationpoints is chosen from E, and denote it by A={α₁,α₂, . . . ,α₁₄}={1,β¹⁷,β^(17·2),β^(17·3), . . . , β^(17·13)}. Then, the 14 codewordsymbols, in the 14 nodes, are

$\left\{ {N_{m} = {{f\left( \alpha_{m} \right)} = {{\sum\limits_{j = 0}^{\alpha}{u_{j}\alpha_{m}^{j}\text{:}\alpha_{m}}} \in A}}} \right\}.$

Algorithm steps are described below according to one or moreembodiments. Assuming the node N=f(α*)fails, α*∈A. The transmittedsymbols from the other nodes (called helpers) are related to the failednode symbol through the dual codeword c1, c2, . . . , c14) of RS(14,10):

${c_{*}N_{*}} = {\sum\limits_{{m = 1},{m \neq *}}^{14}{c_{m}{N_{m}.}}}$

Table V shows column multipliers v_(m) corresponding toA={1,β¹⁷,β^(17·2), . . . β^(17·13)}

TABLE V ν₁ ν₂ ν₃ ν₄ ν₅ ν₆ ν₇ ν₈ ν₉ ν₁₀ ν₁₁ ν₁₂ ν₁₃ ν₁₄ β

β

1 β

β²²¹ β³⁴ β²³⁸ β¹³⁶ β²³⁸ β²²¹ β¹⁰² β¹⁰² β³⁴ 1

indicates data missing or illegible when filed

Table VI shows a dual basis for each node.

TABLE VI Failed node N₁ N₂ N₃ N₄ N₅ N₆ N₇ N₈ N₉ N₁₀ N₁₁ N₁₂ N₁₃ N₁₄ γ₁β²⁰³ β¹¹⁸ β

β²⁰³ β³³ β²²⁰ β¹⁶ β¹¹⁸ β¹⁶ β³³ β¹⁵² β¹⁵² β²²⁰ β²⁵⁴ γ₂ β¹⁵² β

β²⁰³ β

β²³⁷ β¹⁶⁹ β

β⁶⁷ β²²⁰ β²³⁷ β¹⁰¹ β¹⁰¹ β¹⁶⁹ β

γ₃ β

β

β¹³⁵ β⁸³ β¹⁶⁹ β

β¹⁵² β

β

β¹⁶⁹ β³³ β³³ β¹⁰¹ β

γ₄ β

β

β

β¹⁶ β¹⁰¹ β³³ β

β

β

β¹⁰³ β²²⁰ β³²⁰ β³³ β

γ₅ β¹⁸⁷ β¹⁰² β²³⁸ β

β¹⁷ β²⁰⁴ 1 β¹⁰² 1 β¹⁷ β¹³⁶ β¹³⁶ β²⁰⁴ β

γ₆ β¹⁸⁸ β

β¹⁸⁷ β

β²²¹ β¹⁵³ β²⁰⁴ β

β²⁰⁴ β

β⁸⁸ β⁸⁵ β¹⁵³ β¹⁸⁷ γ₇ β

β²³⁸ β

β

β¹⁸³ β⁸⁵ β¹³⁶ β²³⁸ β¹³⁸ β¹⁸³ β¹⁷ β¹⁷ β⁸⁵ β¹¹⁹ γ₈ 1 β¹⁷⁰ β

1 β

β¹⁷ β

β¹⁷⁰ β⁶⁸ β⁸⁸ β

β

β¹⁷ β

indicates data missing or illegible when filed

In particular, a dual codeword corresponds to evaluations of apolynomial over F of degree at most n

k

1=3. Here the evaluations of 8 appropriate dual codeword polynomials arefound in Step 1. In each helper, at most 4 evaluations are linearlyindependent, leading to the bandwidth reduction (Step 2). As in Step 3,we map symbols of F to sub-symbols of the base field, B=GF (2), usingthe trace function:

(x)=x+x²+x⁴ . . . +x¹²⁸, where for every x∈

,

(x)∈

. Taking trace on both sides of (2), one obtains 8 trace functions ofthe failed symbol in Step 4. Finally in Step 5, the following propertyis used to repair the failed symbol. Let {γ₁,γ₂, . . . , γ₈} and{γ₁′,γ₂′, . . . , γ₈′} be dual basis for F over B, then any symbol x∈Fcan be reconstructed from its 8 trace functions: x=Σ_(i=1) ^(s)γ_(i)′

(γ_(i)x)

Repair algorithm for RS (14, 10) over GF(28:

Step 1: For helper node m, such that α_(m)∈A\{α*} evaluate 8 polynomialsat α_(m) {ν_(m)η_(t)p_(j)(α_(m)), t∈[2], j∈[4]}, where νm

s are the column multipliers and provided in Table V,

${\left\{ {\eta_{1},\eta_{2}} \right\} = \left\{ {1,\beta} \right\}}, {{{and}\mspace{14mu} {p_{j,*}(x)}} = {\xi_{j}{\prod\limits_{w \in W}\left( {x - \alpha_{*} + {w^{- 1}\xi_{j}}} \right)}}}, {\xi_{j} = \beta^{17{({j - 1})}}}, {W = {\left\{ {1,\beta^{17},\beta^{68}} \right\}.}}$

Step 2: Polynomial evaluations calculated in Step 1 from the same helperare in a subspace of F with dimension 4. Let us represent a basis ofthis 4-dimensional subspace as {sm1, sm2, sm3, sm4}. If the result ofStep 1 is 4 unique values, directly represent them as {sm1, sm2, sm3,sm4}. If there are more than 4 unique values, there are many differentways to find a basis of the subspace. One general way is to view theresults as vectors over B8 and write them in an 8×8 matrix over B. Then,one can apply the row reduction method to derive a basis in the first 4nonzero rows.

Step 3: Download the binary values D_(m,j)=

*(s_(mj)N_(m)), j∈[4] from node m.

Step 4: Since the trace function is a linear function and is in thesubspace constructed by {sm1, sm2, sm3, sm4}, we use the informationdownloaded from the helpers to represent.

(ν_(m)η_(t)p_(j*)(α_(m))N_(m)) and then calculate

(ν_(*)η_(t)p_(j,*)(α_(*))N_(*)), i∈[2], j∈[4] from

${{tr}_{\text{/}}\left( {\upsilon_{*}\eta_{t}{p_{j,*}\left( \alpha_{*} \right)}N_{*}} \right)} = {- {\sum\limits_{{m = 1},{m \neq *}}^{14}{{{tr}_{\text{/}}\left( {\upsilon_{m}\eta_{t}{p_{j,*}\left( \alpha_{m} \right)}N_{m}} \right)}.}}}$

Altogether, 8 bits are calculated for the failed node N*.

Step 5: {ν_(s)η_(t)p_(j,*)(α_(*)):t∈[2], j∈[4]} forms a basis for F overB. Let us present these 8 values as the set {γ1,γ2, . . . , γ8}. Then,we use the 8 bits calculated in Step 4 to reconstruct N_(*) by

${N_{*} = {\sum\limits_{i = 1}^{8}{\gamma_{i}^{\prime}{{tr}_{\text{/}}\left( {\gamma_{i}N_{*}} \right)}}}},$

where {γ₁′,γ₂′, . . . , γ₈′} is the dual basis of {γ₁, γ₂, . . . , γ₈}for F over B, and are provided in Table VI.

Example

According to one embodiment, a detailed repair algorithm example isprovided in which the failed node is N1.

Step 1: The 8 polynomials ν_(m)η_(t)p_(j,1)(α_(m)), t∈[2], j∈[4] areprovided in Table VII.

TABLE VII THE COEFFICIENTS

 OF DIFFERENT HELPERS WHEN N₂ FAILS. helper node N₂ N₃ N₄ N₅ N₆ N₇ N₈ N₉N₁₀ N₁₁ N₁₂ N₁₃ N₁₄

 = β¹⁷ 1 0 1 β

β

β¹⁰² β

β¹⁷ β¹⁷ β

0 β¹¹⁹ 1(t = 1, j = 1)

 = β¹⁷ β¹¹⁹ β⁶⁸ 0 β⁶⁸ β

β²⁰³ β⁸⁵ β

β⁵¹ 0 β¹⁰² β

2(t = 1, j = 2)

 = β¹⁷ 1 β

0 β¹⁷⁰ 0 β⁶⁸ 0 β¹⁷ β⁵¹ β²³⁸ β¹³⁶ β¹⁷ 3(t = 1, j = 3)

 = β¹¹⁹ β¹¹⁰ 0 β¹¹⁹ β

β¹⁸⁷ β²⁰³ 0 β

β⁵¹ β¹⁷ β²³⁸ β¹⁷ 4(t = 1, j = 4)

 = β¹⁸

0

β

β¹⁸⁸ β¹⁰³ β²³⁹ β¹⁸ β

β

0 β

5(t = 2, j = 1)

 = β¹⁸ β¹²⁰ β⁶⁹ 0 β⁶⁹ β⁶⁹ β²⁰⁵ β⁸⁵ β

β⁵² 0 β¹⁰³ β

6(t = 2, j = 2)

 = β¹⁸

β²⁰⁵ 0 β¹⁷¹ 0 β

0 β¹⁸ β⁵² β

β¹³⁷ β¹⁸ 7(t = 2, j = 3)

 = β¹²⁰ β¹²⁰ 0 β

β

β

β

0 β⁶² β⁵² β

β

β¹⁸ 8(t = 2, j = 4)

indicates data missing or illegible when filed

TABLE VIII THE COEFFICIENTS

 OF DIFFERENT HELPERS WHEN N₁ FAILS. helper node N₂ N₃ N₄ N₅ N₆ N₇ N₈ N₉N₁₀ N₁₁ N₁₂ N₁₃ N₁₄ s_(m1) β¹⁷ 1 β

1 β

β

β

β⁸⁵ β¹⁷ β¹⁷ β

β¹⁰² β¹¹⁹ s_(m2) β¹⁸ β β⁶⁹ β β

β⁶⁹ β²⁰⁴ β⁸⁶ β

β¹⁸ β²³⁸ β

β²³⁸ s_(m3) β¹¹⁹ β¹¹⁹ β²⁰⁴ β³⁵ β¹⁹⁵ β¹⁸⁷ β¹⁰³ β²³⁸ β¹⁸ β⁵¹ β¹²⁰ β¹⁰³β¹²⁰ s_(m4) β¹²⁰ β¹²⁰ β²⁰⁵ β⁶⁹ β¹¹⁶ β¹⁸⁸ β²⁰³ β²³⁹ β

β⁵² β²³⁹ β¹³⁷ β²³⁹

indicates data missing or illegible when filed

TABLE IX

 OF DIFFERENT HELPERS WHEN N

 FAILS. helper node N₂ N₃ N₄ N₅ N₆ N₇ N₈ N₉ N₁₀ N₁₁ N₁₂ N₁₃ N₁₄

 = 1(t = 1, j = 1) D_(2,1) D_(3,1) 0 D_(5,1) D_(6,1) D₇,

D_(8,1) D_(9,3) D_(10,1) D_(11,1) D_(12,1) 0 D_(14,1)

 = 2(t = 1, j = 2) D_(2,1) D_(3,3) D_(4,1) 0 D_(6,2) D_(7,1) D_(8,2)D_(9,1) D_(10,2) D_(11,3) 0 D_(13,1) D_(14,2)

 = 3(t = 1, j = 3) D_(2,1) D_(3,1) D_(4,3) 0 D_(6,1) + 0 D_(8,1) + 0D_(10,3) D_(11,3) D_(12,2) D_(13,2) D_(14,1) + D_(6,2) D_(8,2) 0D_(14,2)

 = 4(t = 1, j = 4) D_(2,)

D_(3,3) 0 D_(5,3) D_(6,1) D_(7,3) D_(8,2) 0 D_(10,1) + D_(11,3)D_(12,1) + D_(13,1) + D_(14,1) + D_(10,3) D_(12,2) D_(13,2) D_(14,2)

 = 5(t = 2, j = 1) D_(2,3) D_(3,2) 0 D_(5,2) D_(6,)

D_(7,)

D_(8,)

D_(9,4) D_(10,3) D_(11,2) D_(12,)

0 D_(14,2)

 = 6(t = 2, j = 2) D_(2,2) D_(3,4) D_(4,2) 0 D_(6,3) D_(7,2) D_(8,3)D_(9,2) D_(10,4) D_(11,4) 0 D_(13,3) D_(14,1) +

 = 7(t = 2, j = 3) D_(2,3) D_(3,2) D_(4,4) 0 D_(6,3) + 0 D_(8,3) + 0D_(10,3) D_(11,4) D_(12,4) D_(13,4) D_(14,2) D_(6,4) D_(8,4)

 = 8(t = 2, j = 4) D_(2,4) D_(3,4) 0 D_(5,4) D_(6,3) D_(7,4) D_(8,4) 0D_(10,2) + D_(11,1) D_(12,3) + D_(13,3) + D_(14,3) + D_(10,4) D_(12,)

D_(13,4) D_(14,2)

indicates data missing or illegible when filed

Step 2: (sm1, sm2, sm3, sm4) are tabulated in Table VIII.

Step 3: Download

(s_(mj)N_(m)), j∈[4].

Step 4: {

=(ν_(m)η_(t)p_(j,1)(α_(m))N_(m)), t∈[2], j∈[4]} are calculated in TableVIX, and {

(ν₁η₁p_(i,1)(1)N₁), t∈[2], j∈[4]} are calculated from (3).

Step 5: We can calculate N1 from (4) and Table VI

As such a detailed algorithm is provided to implement the repair of theRS(14, 10) code over GF(2⁸) in Facebook

s f4 system. Our algorithm can repair one failed node and requires only4 bits from each helper.

While this disclosure has been particularly shown and described withreferences to exemplary embodiments thereof, it will be understood bythose skilled in the art that various changes in form and details may bemade therein without departing from the scope of the claimedembodiments.

What is claimed is:
 1. A method for error correction of distributed dataincluding modifying a Reed-Solomon correction code, the methodcomprising: receiving, by a device, a first correction code for aplurality of data fragments stored in a plurality of storage nodes,wherein the first correction code is a Reed-Solomon code having a datasymbol for each the plurality of storage nodes and wherein the firstcorrection code is represented as a first polynomial over a first finitefield having a first subpacketization size; constructing, by the device,a second correction code in response to at least one unavailable storagenode of the plurality of storage nodes, wherein the second correctioncode is represented as a second polynomial over a second finite field,the second polynomial having an increased subpacketization size relativeto the first polynomial; performing, by the device, erasure repair forthe at least one unavailable storage node using the second correctioncode, wherein the second correction code is applied to available datafragments of the plurality of data fragments for erasure repair, whereinthe erasure repair utilizes at least one coset of a multiplicative groupof the second finite field; and outputting, by the device, a correcteddata fragment based on the erasure repair.
 2. The method of claim 1,wherein the second correction code is constructed for single erasurerepair and erasure repair utilizes at least one of one coset and twocosets, wherein code length (n) and dimension (k) of the firstcorrection code are maintained.
 3. The method of claim 1, wherein thesecond correction code is constructed for single erasure repair anderasure repair utilizes multiple cosets, the code length (n) of thesecond correction code is increased and redundancy (r) is fixed.
 4. Themethod of claim 1, wherein the second correction code is constructed forsingle erasure repair, the second correction code providing a scalarcode with evaluation points selected from one coset.
 5. The method ofclaim 1, wherein the second correction code is constructed for singleerasure repair with evaluation points in selected two cosets, anderasure repair includes selection of correction code polynomials thathave a full rank condition in a coset of an unavailable node and rank 1when evaluated at another coset.
 6. The method of claim 1, wherein thesecond correction code is constructed for single erasure repair, whereinevaluation points for erasure repair are chosen from multiple cosets toincrease code length.
 7. The method of claim 1, wherein the secondcorrection code is constructed for repairing multiple erasures in theplurality of data fragments, wherein evaluation points for erasurerepair are in at least one of one coset and multiple cosets.
 8. Themethod of claim 1, wherein the second correction code is constructed forrepairing multiple erasures in the plurality of data fragments, whereinerasure repair includes use of a helper node to reconstruct at least onesymbol for a first data fragment, and wherein the reconstructed symbolis used for erasure repair of a second data fragment.
 9. The method ofclaim 1, wherein the performing of the erasure repair includes using afirst dual codeword and a second dual codeword as repair polynomials,and combining a trace function with dual codewords to generatefragments.
 10. The method of claim 1, further comprising determining anerasure repair scheme for the plurality of data fragments.
 11. A deviceconfigured for error correction of distributed data including modifyinga Reed-Solomon correction code, the device comprising: an interfaceconfigured to receive a first correction code for a plurality of datafragments stored in a plurality of storage nodes, wherein the correctioncode is a Reed-Solomon code having a data symbol for each the pluralityof storage nodes and wherein the first correction code is represented asa first polynomial over a first finite field having a firstsubpacketization size; a repair module, coupled to the interface,wherein the repair module is configured to: construct a secondcorrection code in response to at least one unavailable storage node ofthe plurality of storage nodes, wherein the second correction code isrepresented as a second polynomial over a second finite field, thesecond polynomial having an increased subpacketization size relative tothe first polynomial; perform erasure repair for the at least oneunavailable storage node using the second correction code, wherein thesecond correction code is applied to available data fragments of theplurality of data fragments for erasure repair, wherein the erasurerepair utilizes at least one coset of a multiplicative group of thesecond finite field; and output a corrected data fragment based on theerasure repair.
 12. The device of claim 11, wherein the secondcorrection code is constructed for single erasure repair and erasurerepair utilizes at least one of one coset and two cosets, wherein codelength (n) and dimension (k) of the first correction code aremaintained.
 13. The device of claim 11, wherein the second correctioncode is constructed for single erasure repair and erasure repairutilizes multiple cosets, the code length (n) of the second correctioncode is increased and redundancy (r) is fixed.
 14. The device of claim11, wherein the second correction code is constructed for single erasurerepair, the second correction code providing a scalar code withevaluation points selected from one coset.
 15. The device of claim 11,wherein the second correction code is constructed for single erasurerepair with evaluation points in selected two cosets, and erasure repairincludes selection of correction code polynomials that have a full rankcondition in a coset of an unavailable node and rank 1 when evaluated atanother coset.
 16. The device of claim 11, wherein the second correctioncode is constructed for single erasure repair, wherein evaluation pointsfor erasure repair are chosen from multiple cosets to increase codelength.
 17. The device of claim 11, wherein the second correction codeis constructed for repairing multiple erasures in the plurality of datafragments, wherein evaluation points for erasure repair are in at leastone of one coset and multiple cosets.
 18. The device of claim 11,wherein the second correction code is constructed for repairing multipleerasures in the plurality of data fragments, wherein erasure repairincludes use of a helper node to reconstruct at least one symbol for afirst data fragment, and wherein the reconstructed symbol is used forerasure repair of a second data fragment.
 19. The device of claim 11,wherein the performing of the erasure repair includes using a first dualcodeword and a second dual codeword as repair polynomials, and combininga trace function with dual codewords to generate fragments.
 20. Thedevice of claim 11, further comprising determining an erasure repairscheme for the plurality of data fragments.